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18th Annual Conference on 
Distance Teaching and Learning 

August 14-16, 2002 
Madison, Wisconsin 

This year the Annual Conference on Distance Teaching and Learning celebrates its 18th anniversary in 
serving as a forum for the exchange of information on distance education and training. In the past decade 
and a half, thousands of attendees have gathered in Madison to discuss key issues in using technology 
effectively. And hundreds of speakers have shared their knowledge and experience through papers, 
presentations, workshops, and roundtables. 

Throughout its 18-year history, the conference’s major aim has been to increase our understanding of factors 
that contribute to effective applications. While emerging technologies continue to offer new promise, the 
academic and corporate environments that use them are constantly buffeted by a variety of cultural and 
economic forces. Each year the conference brings together practitioners who creatively deal with the dynamic 
of adapting new technologies to changing distance education settings. This makes the conference experience 
and the resources it produces a unique view of the current state of affairs in distance education practice. 

As in the past the keys to success reside in human factors — in what people do to plan, design, and support 
distance learning applications. This year’s conference is no exception. The 18th Annual Conference 
proceedings contains 90 papers that address important human factors from several perspectives, including: 
learning environments, course development, blended delivery technologies, instructional design, faculty 
development, evaluation and research, learners and learner support, planning and management, and teaching 
methods. The authors, as experienced distance educators or trainers, provide insights into how those factors 
contribute to successful outcomes and describe practical methods for implementing similar approaches in 
other settings. 

The field of distance education and e-leaming is changing to meet new challenges with fresh approaches 
using promising new technologies. This year’s conference focused on these issues with an expanded 
program made possible by our new location at the Monona Terrace Convention Center. Distance Learning 
2002 provided more than 140 workshops, seminars, keynotes, information sessions, course showcases, and 
in-depth forums highlighting topics of interest for both new and experienced educators 

My thanks to the many presenters who have contributed to this conference over the past 1 8 years. They are 
the heart of the conference. This book contains a current sampling of the rich sharing of information upon 
which the conference has built its reputation. The conference staff and planning committee look forward to 
expanding the availability of these resources to practitioners in the field by electronic means throughout the 
year as well as gathering new presentations for next year’s conference. 

William Winfield 
Conference Director 
University of Wisconsin-Madison 
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Using Students’ Learning Styles to Provide 
Support in Distance Education 

Mohamed Ally, Ph.D. 

Associate Professor, Athabasca University 

Patrick Fahy, Ph.D. 

Associate Professor, Athabasca University 

There are many learning style instruments that are used to determine students’ learning styles. The two 
most widely used in education are the Kolb Learning Style Inventory (1 984) and the Myers-Briggs Type 
Indicator (Myers, 1978). The Kolb Learning Style Inventory (LSI) looks at how learners perceive and 
process information while the Myers-Briggs Type Indicator uses dichotomous scales measuring 
extroversion versus introversion, sensing versus intuition, thinking versus feeling, and judging versus 
perception. 

The first component in the LSI, perceiving, is the way we perceive and absorb the information around us. 
This can range from concrete experience to abstract conceptualization. Concrete experience relates to 
one’s preference to learn things that have personal meaning in life. The second dimension of perceiving 
is reflective observation. Students who prefer this style like to take the time to think and reflect on the 
learning materials. The second component, processing, is related to how one understands and processes 
the information that is absorbed after perceiving the information. Processing ranges from abstract 
conceptualization to active experimentation. Learners who have a preference for abstract 
conceptualization like to learn facts and figures and like to research new information on different topics. 
Learners who have a preference for active experimentation prefer to apply what they learn to real life 
situations. They like to try things and learn from their experience. 

The distance education instructor should include a variety of support strategies for the different styles to 
allow learners to choose the appropriate strategy based on learning style. Concrete experience learners 
prefer experiences in which they can be involved and they relate to peers and not to people in authority. 
They like group work and peer feedback and they see the distance education instructor as coach or helper. 
These learners prefer support methods that allow them to interact with peers and obtain coaching from the 
instructor. Reflective observation learners like to observe carefully before taking any action. They prefer 
passive delivery methods and see the instructor as the expert. They tend to avoid interaction with 
studoits. In terms of support, the reflective observation learners need the most direct support from the 
instructors since they see the instructor as the expert. Abstract conceptualization learners like to work 
with things and symbols and less with people. They like to work with theory and to conduct systematic 
analysis. Support from the instructor takes the form of coaching. Active experimentation learners prefer 
to learn by doing practical projects and group discussions. They like active learning methods and like to 
interact with peers for feedback and information. Th^ tend to establish their own criteria to evaluate 
situations and things. For the Abstract conceptualization learners, the instructor should provide 
opportunities to learn by discovery and be behind the scenes during the learning process. Some initial 
support is required but the amount of support may drop considerably during the learning process. 

Procedure 

The subjects for this study were students enrolled in two courses in a graduate program at a distance 
education institution. The Kolb Learning Style Inventory (LSI) was administered to students in the two 
courses. The LSI was used to determine students’ learning styles. Thirty-nine students (26 females and 
13 males) completed the LSI. 
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The support strategies in the course included computer conferencing, e-mail, and telephone 
communication. There were multiple computer conferences in the course where students exchanged ideas 
and the instructor acted as a moderator for the conferences. Participation in the conferences was 
mandatory and students were assigned marks for contributing to the conferences. 

Results 



Learning Style Distribution 

Data from the analysis revealed an uneven distribution of learning styles for students (Figure 1). The 
largest percent of students were Assimilators (39%), followed by Convergers (33%), Accommodators 
(18%), and Divergers (10%). 



Figure 1; Learning Styie Distribution 




Diverger Assimilator Converger Accommodator 



Mean Number of Computer Conference Comments by Learning Styles 

One of the support strategies used in the graduate courses is computer conferencing. There were multiple 
computer conferences in which students had to respond to original discussion questions and provide 
comments to other students’ comments. The computer conferences allowed for peer-to-peer, student- 
instructor, and instructor-student communication. The instructor acted as a moderator by responding to 
questions and comments, keeping the discussion on track, starting and ending each conference, and 
evaluating students’ performance in the conferences. 

Figure 2 shows the mean number of comments for all of the conferences by learning style. Convergers 
contributed more (M = 22) to the conferences than Divergers (M = 14.5). This was followed by 
Assimilators (M = 16.26) and Accommodators (M = 16.14). Convergers tend to try new things and like to 
solve problems. They are practically oriented and they like to test ideas before accepting them. One 
reason that Convergers provided the largest number of comments in the conferences could be because 
they like to test their ideas out with other students. As a result, they posted their ideas to get reaction from 
other students. On the other hand. Divergers tend to be people persons and are oriented to feelings. As a 
result, they may have found the non face-to-face conference discussion to be uncomfortable and it was 
difficult for them to connect with others. 
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Figure 2: Mean # of Comments by Learning Styie 
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Mean Number of Original Threads by Learning Style 

In all of the computer conferences, the instructor posted the original discussion question and students 
were required to respond to this original posting; however, some students ignored the original discussion 
questions and started their own threads. As shown in Figure 3, the mean number of original threads was 
the highest for Accommodators (M = 1.85), followed by Assimilators (M = 1.66), Convergers (M = 0.76), 
and Divergers (M = 0.50). Accommodators tend to take a leadership role and are risk-takers. As a result, 
they took the initiative to start their own threads. On the other hand. Divergers had the lowest number of 
original postings. They are people oriented and they tend to respond to other students’ comments and to 
provide support to other students. Hence, Divergers are less likely to start new threads. 



Figure 3: Mean # of New Threads by Learning Style 
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Level of E-maU Interaction with Instructor 

For one of the courses in this study, the number of e-mail interactions between the instructors and 
students was tracked. The level of interaction was placed into three categories: High level of e-mail 
interaction, defined as ten or more e-mail interactions between the student and instructor; Medium 
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interaction, consisting of between five and nine e-mails; and low interaction, less than five e-mails. Data 
analysis revealed that assimilators had the highest number of interactions with the instructor (Figure 4). 
This is not surprising since assimilators see the instructor as the expert and prefer to obtain information 
directly from the instructor. They interact less with their peers and more with the instructor. They are 
also goal oriented and like to interact with the instructor to set expectations for assignments, exams, and 
other course requirements. 



Figure 4: Level of E-mail Interaction With Instructor 
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Conclusion 

Based on the results of the study, it appears that Assimilators require the most support in a distance 
education course. They see the instructor as an expert; hence, they like to interact with the instructor 
rather than other students. Divergers and Accommodators may require less support since they tend to rely 
more on other students rather than the instructor, and they also interact with other people outside of the 
course. Analysis of computer conference participation showed Convergers being the most active while 
Divergers were the least active. One explanation for this result is Convergers like to share and test their 
ideas with others students while Divergers are uncomfortable sharing their ideas in the conferences. 
Divergers may prefer the face-to-face approach. 

Distance educators must ensure that adequate support strategies are provided for students with different 
learning styles. Catering to the different learning styles could result in higher retention in distance 
education. 
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Is Your E-everything Accessible to Everyone? 
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Getting an Overall View of a Web Page 

Imagine viewing a Web page made up of one giant block of text. There are no headers, no breaks for 
paragraphs or list items, and no differences in font size or color. Of course, no one would design a page 
that would look like this, although some of the pages I have visited come close. If you are using a small 
screen device such as a cell phone or palm pilot to view a page, certain isolated parts of the page may 
look totally unformatted. 

Now, imagine that you are using a full screen device and the page looks a lot better. You can see major 
headings, paragraphs, and bulleted lists are obvious. The changes in font size and the use of color make 
the page quite understandable. You can determine the information you want to read first with a single 
glance. 

Suppose, however, that you just returned from the eye doctor who put drops in your eyes, which made it 
hard for you to see for an hour or two. You need to read some information on a Web page so you ask a 
co-worker to read it to you aloud. The person can only read one word at a time, and they do not announce 
any of the visual elements on the page such as list items, headers, etc. You want to skip ahead, but you 
don’t know anything about the part of the page you haven’t read, so you can’t ask the person reading to 
go to a particular place. You want to go back and review something you have read, but you can’t quite 
remember what was said, so again, it’s hard to ask the reader to go back to a certain point. 

In fact, this is the way persons who use screen readers deal with Web pages every day. The synthesized 
voice can only read one word at a time. How then does the screen reader user know how to skip to or go 
back to something of importance to her? Of course, even with your temporary disability, you could ask 
your reader to make the page much more meaningful to you, you by telling you when they reached a new 
header, paragraph, list item, etc. And that’s exactly what screen reader users need. The nice thing is, it 
takes almost no work at all to make it happen. Making it happen for screen reader users also makes it 
happen for people who use small screen devices. 

Emerging Technologies and Device Independence 

“Mobile devices and audiovisual media are changing the way people are interacting with the Web. Web 
technology can accommodate these innovations, since, from the start, the Web has been conceived as a 
universal system for distributing and accessing information” (Philipp Hoschka, W3C World Wide Web 
Consortium, Interaction Domain Leader). 

Imagine accessing a complex Web page (or pages) with a cell phone or other text only device? It can be 
pretty disorienting and often fiiistrating to try to orient oneself to a complex page. In the not-too-distant 
past, Web designers were concerned with cross-platform and cross-Browser compatibility. Certainly, this 
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is still important when considering the end-user environment. However, accessing information on the 
Web is no longer limited to the desktop computer. New and often ingenious devices such as cell phones, 
Personal Data Assistants, Web tablets and a variety of everyday appliances such as TVs, are allowing for 
access to the Web. As the user-base of these new devices increases, user expectations and demands for 
access (from multi-device and multi-location) will also increase. 

It is safe to assume that for many users a high-bandwidth, high-resolution and multimedia-enabled 
desktop will continue to serve as the primary device for access. However, evolving technologies are 
allowing users to make an access choice, largely based on their environment or situation. Access needs 
are not limited to a controlled or optimum environment. Some users will find themselves wanting to 
access information under restrainal or limiting circumstances, that is, in a noisy environment, in a sight- 
limited environment, or while in a hurry. Others may want to access the Web in a more free-form manner, 
for example, using emerging devices and even using hands-free and/or eyes-free devices. 

The key to successful Web design is to design for flexibility and inclusion so that you do not exclude any 
particular audience; making the Web accessible to anyone, anywhere, anytime, anyhow. The Web 
Content Accessibility Guidelines (WCAG) available via the W3C World Wide Web Consortium, Web 
Accessibility Initiative, help to ensure that users can access your information regardless of device, 
situation or ability. In the continually evolving world of Web applications and Web technologies it makes 
sense to design your content and service so that it can be adapted quickly and efficiently to meet any new 
circumstance. Using existing design techniques like those in WCAG 1 .0 will ensure that your message 
will be readily available to your changing (and expanding) customer base and new technologies they may 
choose. 



Access Barriers 

Increasing access to e-everything can be argued on ethical, legal and economic grounds. It's the right 
thing to do. There are legal mandates that state accessible means must be provided for Internet-based 
programs, goods or services. There are varying estimates of the dollar amount people with disabilities 
control, and those estimates continues to rise as the population in general ages — ^who can afford to ignore 
this revenue source? We also know that those accessing the Web using hand-help or new devices is also 
growing. We just cannot afford to ignore or exclude any audience. 

Assistive technologies in the form of specialized hardware and software products allow individuals with a 
wide range of abilities and disabilities to use computing and networking technologies. These technologies 
do not remove all access barriers however. Planning for access when e-everything is being developed is 
much easier than creating accommodation strategies or retrofitting after the fact. Disabilities, age, race, 
native language, ethnicity, gender are a few of the potential characteristics that should also be considered 
when designing e-everything. Let's take a look at these. 

Blind Internet users may be using a computer equipped with screen reader software and a speech 
synthesizer, which allows the text that appears on a screen to be read with a synthesized voice. A text- 
only browser or a Braille refreshable display that prints the text line by line. Images (including 
photographs, image maps, animations, symbols, scripts and images used a list bullets, spacers, graphical 
buttons etcetera) can not be read or interpreted by blind internet users unless text alternatives are 
provided. 

Other visual impairments may limit the amount of Web page seen when special software to enlarge 
screen images and text are used. When pages are cluttered or the layout changes from page to page, the 
user can become confused or disoriented. Colorblind users cannot successfully navigate or understand 
content when it is necessary to distinguish between colors. 
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Deaf and hard of hearing individuals may benefit from Internet resources when th^ do not require the 
ability to hear. When audio output is required, text captioning or transcription is necessary (including 
videoconferences and streaming). 

Speech impairments and deaf or hard of hearing individuals may experience barriers to interactive 
participation, requiring alternative means such as electronic mail, text telephone (TTY), or 
Telecommunications Relay Service (TRS). 

Seizure disorders can induce seizures when Web pages include attention-grabbing features or flicker 
(often between 2 to 25 hertz). 

Mobility impairments may prevent individuals from moving their hands. Alternative keyboard and 
mouse, or speech input could be used to gain access to Internet-based information or communication. 
Others may lack fine motor skills required to select small buttons on the screen. Individuals with slow 
response rate may also be unable to effectively participate in real-time "chat" communications. 

Learning disabilities may impact an individual's ability to read, write, and process information. Speech 
output and/or screen enlargement systems used by individuals with visual impairments may also be used 
by those with learning disabilities. It is also difficult for some individuals with learning disabilities to 
understand Web sites when the information is cluttered or when the screen layout changes from one page 
to another. 



Plan for Access 

Considering the wide range of abilities, and Web access devices that exist as your ©-everything is being 
planned and developed is much easier than creating accommodation strategies or retrofitting your site 
after the fact. Benefits extend beyond people with disabilities and those who have the latest nifty gadgets. 
For example, individuals with temporary disabilities and/or situational limitations, age, race, native 
language, ethnicity and gender differences also benefit. A person with computer system limitations is in a 
similar situation as a person who is blind. A monochrome monitor recreates a limitation that a colorblind 
person experiences. People multi-tasking (operating a computer and their hands are occupied with other 
activities) face challenges similar to those who use hands-free input methods because of a disability. 

People who have turned off their images because of slow modems, or because they want their pages to 
load faster, need text equivalents for all images in the same way that blind users do. A noisy environment 
that interferes with audio creates access barriers similar to those experienced by individuals with hearing 
impairments. When English is not the first language, reading difficulty similar to the experiences of some 
individuals with learning disabilities can occur. Providing multiple formats of information also addresses 
differences in learning styles. 

Don't wait until someone asks about accessing your e-everything content, or worse yet, being a participant 
in a lawsuit because your site could not be accessed. Be proactive. Think about the wide range of abilities, 
disabilities and devices that exist at the design stage. Include information on your site about who to 
contact if someone is having difficulty accessing information on your site. Test your site using hearing or 
sight alone; navigate without using the mouse, use different browsers (including a text only browser), and 
with different validation tools (listed at the W3C World Wide Web Consortium site). 

Use valid / correct html. Some screen readers allow the user to bring up a list of the major headers on 
the page. This basically gives the screen reader user the same ability you have to quickly scan the page 
and find the part of the material you want to read. But this only works if headers are tagged with the 
correct html code. Some screen readers announce items in a list with a different voice, but again, this 
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only works if the correct html code for list items is used. In the future, it may well be that cell phones and 
palm pilots give the user some easy way to go to major sections of the page in order to compensate for the 
inability to see enough of the page to visually get an overall view of the layout. Again, devices can only 
do this if the correct html code is used. 

Images and animations need to have a text equivalent (alt attribute / alt-text) to describe the function of 
each visual. Likewise, text used to describe links needs to be functional without being either too short to 
have any meaning or too long to allow clear comprehension. So, a link that simply says “Here,” or “Click 
Here,” wouldn’t work if you were just browsing the link titles. If you went to a link that simply said 
“Research,” you may not be thrill^ to read a two-paragraph synopsis of the research only to find that it 
was not something you needed to take the trouble to click on. You might wonder why the link couldn’t 
have said something like “Research on using computers by persons who are aging.” Animations, 
multimedia, videos, steaming etc need captioning and transcripts of audio and descriptions of the video. 

If you go to a page with a screen reader, a small screen or text-based browser, the alt-text that is used 
needs to give you the same ability to operate as you might if the graphics were visible. You would have a 
much more enlightening experience if the alt-text captured the function, and not just the description, of 
the graphical elements you can no longer see. So, if you were watching a slide show on a site you may 
have some difficulty figuring out what to do if the two alt-text labels simply said “Stylized arrow keys.” 
You would, no doubt find it more helpful if the alt-text announced each arrow as, “Previous slide,” and 
“Next slide.” 

Stay current with Web changes. The Web, as well as the tools and devices that create and access 
information on the Web continue to change. The Web is considered a necessary part of some people's 
employment, and a key tool for gaining information, services and goods for others. We've seen the Web, 
tools that create and access the Web and legislation or guidelines change at rates that boggle our minds. 
How can we keep up with it? “Just what direction is the Web headed? For the answers to these and other 
general Internet or markup language specific questions, tune your browser to http://www.w3.org/ for a 
vast and varied selection of information. Because the W3C establishes recommendations concerning the 
Web, this site offers interesting possibilities as to future directions for the Web. While the W3C does not 
exercise the influence of an official standards setting organization, they have been influential in bringing 
together industry members and other interested parties to develop solutions and circulate 
recommendations to the public and members.” (Kim Moorman). 
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Sometimes a Good Idea Isn’t Enough: 
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Introduction 

The Kansas City Regional Access Consortium for Higher Education (KC REACHE) is a unique 
collaboration among 10 colleges and universities and the local public television station. This consortium 
enables collaboration and innovation between distance learning providers in the Kansas City region, 
noteworthy because it crosses not only institutional boundaries, and boundaries between community 
colleges and four year schools, but also crosses state lines. The multi-lateral exchange of ideas between 
universities has enabled an accelerated rate of articulation agreements between these institutions, and has 
allowed the schools to weave together different student service offerings enhancing the overall learning 
experience of the Kansas City area distance student. The 1 1 members of the consortium include Avila 
College, Johnson County Community College, Kansas City Kansas Community College, Metropolitan 
Community Colleges, Missouri Western State College, Northwest Missouri State University, Park 
University, Rockhurst University, University of Missouri-Kansas City, Washburn University and KCPT, 
Kansas City’s Public Television station. 

Addressing the Needs of Distance Learners in the Greater Kansas City Area 

A sense of disconnection between the student and institution is a fairly common problem within distance 
learning. The Internet may bring education closer to students, but in some ways it pushes the institution 
farther away. Through collaboration and cooperation, the KC REACHE consortium was able to address 
this issue in the Kansas City area by offering two sets of conveniences to the online learner; education is 
delivered to the student’s desktop via the Internet, and the physical campus is as close as the nearest 
member school, via the institutions network of shared services, including inter-library services, and 
shared testing services. 23 online degree programs and over 500 classes are offered among the 10 
member campuses. 

Research indicates that some of the most important educational forces in higher education are faculty 
members and other students (Pascarella & Terenzini, 1991). The Council for 

Adult and Experiential Learning (2000) notes that if an institution aspires to be adult learning focused, it 
must devise ways to build ‘community’ among its learners. KC REACHE is attempting to explore options 
for creating these connections for students in a virtual world — grounded within the Kansas City 
geographic community. 

Lessons Learned: Creating and Refining Student Support Services for the Distance Learner 

The on-ground model for student services cannot merely be replicated for online students. When KC 
REACHE began to devise solutions to these problems, we decided to borrow the concept and the phrase 
‘brick and click’ from the business world. Arthur E. Levine, president of Teachers College at Columbia 
University, believes that ‘brick and click’ will become the dominant model in higher education (The 
Chronicle, October 27, 2000) based on the idea that those most likely to succeed will be using combined 




Aquino & Wilkinson ❖ 15 



face-to-face services at physical locations (‘brick and mortar’) with the convenience of online services 
(‘point and click’). 

Occasionally, the concept cannot mature beyond the planning stage, due to logistics or lack of resources. 
Odds are, the initial idea will undergo several iterations and will morph dramatically, resulting in a 
program significantly different from the initial concept. In this case, as a result of funding from a FIPSE 
grant, the KC REACHE Brick and Click Project is becoming a reality. In the fall of 2001, KC 
REACHE’s model for shared student support services was validated on a national level, upon the receipt 
of funding from the U.S. Department of Education, through a three-year FIPSE grant in the amount of 
$577,957. With this Brick and Click grant, KC REACHE’s ability to expand its branch campus model 
for shared services was exponentially accelerated. 

As a regional consortium consisting of 10 separate and competing institutions, multiple stakeholders with 
multiple perspectives and policies and procedures worked together to tailor and refine the service 
offerings. Student Services for online students implemented to date: 

1 . Inter-Library Loan Program: Distance students have reciprocal borrowing privileges at 9 different 
campus libraries. 

2. Online Library Resources: Online students can access a digital collection of publications via the 
library website 

3. On-ground Testing Network: If required to take a proctored exam, online students may schedule an 
exam at any of the 10 member institutions testing centers. 

4. Student Organizations Websites: Three student organization websites have been implemented to 
encourage online participation within traditionally on-ground student groups. 

5. Student events: Website established ambling access for online students at on-ground events at all 10 
member institutions. 

6. Student Leadership Group/Mentoring Service: A purely online student organization for distance 
learning students has been created. Members from this group will serve as mentors for new online 
students within the KC REACHE consortium. 



Strategies for the Successful Implementation Student Support Services for Distance Learners 

Once the concept has been formalized, a process for implementing the service program must be created, 
generating buy-in at the highest level of your academic institution, with the critical people within the 
institution involved in the process from the onset. 

One of the biggest challenges is identifying the right people within your organization to get things done; 
an individual with the authority to sign off on the initiative and the ability to follow through and make 
sure it is done 

In the case of KC REACHE, the high-level buy-in is not the issue. Student service deans from our 
member institutions meet regularly, our Steering Committee is comprised of Deans and provosts, and the 
Presidents at each college sit on the Board of Directors. 

When it comes down to the actual implementation, however, more often than not, it is student services 
staff that will work with the program, not the administrators. We are in the process of creating a formal 
plan that will define the roles and responsibilities of each individual for the duration of the project, and 
are developing a framework based on the expectations and goals of the group. 
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So You’ve Built It: Now, Where Are the Students? 

The idea and implementation of student services programs, themselves, are not enough to ensure the 
success of the program. How can you ensure that distance students not only know about the services 
available to them, but are also encouraged to take advantage and participate in the programs? 

For example, several years ago, Kirkwood Community College was awarded FIPSE grant to create 
advising and tutoring services for their online students. Michele Payne, the project’s director, detailed 
some of the issues she encountered while trying to launch these services. Payne saw that few students 
were taking advantage of their services, no matter how much their group “marketed, memo-ed and 
cajoled” (Payne, 2002). She suggests that just offering the support services to students will not 
necessarily result in service usage. In retrospect, she recommends colleges “rely on sticks, not carrots” to 
motivate their students to take advantage of the services put in place to benefit their performance. 
Similarly, after the first year of the Brick and Click Project, KC REACHE has encountered a lower than 
anticipated student participation rate, and we are not attracting the number of students we need to help us 
determine what is working and what is not. 

In order to get the word out to distance students, each member of KC REACHE has posted a link to the 
KC REACHE website which can be accessed through the online class homepage at each institution. In 
addition, KC REACHE has hosted several faculty development workshops. At each workshop, faculty 
members are encouraged to inform their students of the services available through KC REACHE. 

The Road Ahead 

KC REACHE is approaching the second year of the three-year, FIPSE-fiinded ‘Brick and Click’ project. 

Based on our first annual survey completed by participating online students, preliminary impressionistic 
results support our original hypothesis that adult online students are primarily interested in services that 
help them advance in their academic and career goals, as opposed to purely social student activities. 

We have identified the need for a detailed project plan. We are in the process of creating a planning 
document with the Student Services Deans at each institution, where roles and responsibilities will be 
clearly communicated, allowing them to effectively and efficiently delegate activities to the appropriate 
representative at their college or university. 

KC REACHE will use the lessons learned in our first year, coupled with the feedback gathered from our 
surveys to continue to tailor and refine our online services to meet the needs of our students. We will also 
focus our attention on creating a sustainable model, to ensure the continuation of our successful student 
services beyond our grant-funded period. 
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Commuiiication in the (Online) Classroom 

Approximately 20 years before online learning emerged, scholars in two different fields made similar 
statements about the learning experience. In 1972 a doctoral student in adult education at the University 
of Wisconsin researched numerous educational programs where the instructor and student were physically 
separated and developed what is now called the transactional distance theory (Moore Kearsley, 1996, 
pp. 199-203). Moore postulated that distance was a pedagogical, not a geographical, phenomenon. He 
also stated that one could overcome this distance through effective dialogue (i.e., instructor-learner 
interaction) and instructional design (i.e., structure). In a similar vein, three communication scholars 
authored a book in which they declared, “There is a difference between knowing and teaching and that 
difference is communication in the classroom” (Hunt, Scott, & McCroskey, 1978, p. 3). In other words, 
they concluded that the process of communication is at the heart of the learning experience. 

These education and communication scholars understood that the learning experience wasn’t primarily 
about the shape of the room, color of the chalkboard, type of font used for the syllabus, high-tech or low- 
tech equipment in the classroom, or the physical separation between the instructor and students (or the 
students from one another). Rather, the communication dynamics within the classroom environment were 
the critical factor in the learning experience. This is especially significant when considering distance 
education, where the call is often to develop directed learning opportunities in an independent learning 
environment. However, an independent learning environment is not the same as an isolated learning 
environment. One the benefits of online distance education that makes it superior to the older 
correspondence models, is that students can still learn in an academic community, albeit a virtual one. 

Some might argue that distance education, by its very nature, requires self-motivated students to work 
well and therefore the students should be responsible for creating such interaction events. After all, some 
instructors within the traditional model simply show up for class, deliver their lecture to a sea of note- 
taking students, and then leave. Besides, it is typically the distance learner who is isolated, and not the 
instructor who is usually located on a university campus. Failure to address the social and relational 
dynamics within online courses may result in greater feelings of isolation among the distance learners, 
r^uced levels of student satisfaction, poor academic performance, and ultimately increased attrition. 
Successful online facilitators face the challenge of finding ways to connect with students and for students 
to connect with other students in meaningful ways. More often than not, most students wait for the 
professor to “do” something that magically knits or binds them with others in meaningful ways. As a 
result, we believe that online learning puts an added responsibility on the instructor to foster a 
communication-rich learning environment. An understanding of social dynamics, including instructor 
immediacy and classroom community, can assist online instructors as they seek to develop the communal 
scaffolding necessary to support an effective learning environment. 
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Social Dynamics 



Mehrabian (1969) developed the concept of immediacy, which he defined as “those communication 
behaviors that enhance closeness to and nonverbal interaction with another” (p. 203). Someone who 
demonstrates high immediacy behaviors such as maintaining eye contact during a conversation is likely to 
produce a more favorable affect and thus communicate more effectively. Immediacy has often been 
examined as a nonverbal construct marked by behaviors such as leaning toward another, assuming a 
position close to another, and looking into another’s eyes. Verbal behaviors can also contribute to a sense 
of immediacy Similar to nonverbal immediacy, the use of verbal immediacy behaviors such as asking 
questions, using humor, addressing individuals by name, and initiating discussion increases psychological 
closeness. 

A limited number of researchers have attempted to examine the role of instructor immediacy (or, more 
typically, social presence or high interactivity) in the online classroom. Gunawardena (1995) concluded 
that online communication can be perceived as interactive, active, interesting, and stimulating by 
conference participants, depending on the kind of interactions and sense of community by the 
participants. Furthermore, Gunawardena and Zittle (1997) found that social presence alone accounted for 
60% of the explained variance when examining overall participant satisfaction in a computer-mediated 
conferencing environment. Vrasidas and Mclsaac (1999) found that interaction was influenced by the 
structure of the course, class size, feedback, and prior experience with computer-mediated 
communication. One example of the negative effects of nonimmediacy was evident in their studies of 
feedback. “Students felt that the lack of immediate feedback in the online portion of the course was 
discouraging and contributed to their limited participation in the online discussions” (p. 33). They 
advised that teachers provide timely feedback to ensure high levels of interaction in the course. 

Harasim (1990) stated that the computer-mediated classroom is an environment rich with the potential for 
interaction and collaboration among the participants. Palloff and Pratt (1999) declared, “Even in this 
virtual or electronic community, educators must realize that the way the medium is used depends largely 
on human needs . . . these needs are the prime reason that electronic communities are formed” (p. 23). 
They proposed seven basic steps for developing academic communities in online distance classes: clearly 
define the group’s purpose, create a distinctive online gathering place, promote effective leadership from 
within, define norms and a code of conduct, allow for a range of roles, permit and facilitate subgroups, 
and permit students to resolve their own disputes (p. 24). Since the word community comes from the root 
communicare (“to share”), they suggest creating collaborative learning activities to enable students to 
share common experiences during the course of the online class. 

McLellan (1999) postulated that the development of online communities would strengthen the online 
learning experience by fostering a sense of social presence among course participants (p. 40). Moller 
(1998) similarly encouraged the development of learning communities in asynchronous online courses. 

She stated that “the potential of asynchronous learning can only be realized by designing experiences and 
environments which facilitate learning beyond the content-learner interaction. To that end, it becomes 
necessary to create learner support communities” (pp. 115-116). Such online learning communities 
provide a fi-amework for social reinforcement and information exchange while girding the learning 
experience with academic, intellectual, and interpersonal support (p. 1 1 6). 

Developing Communal Scaffolding 

Communal scaffolding recognizes that successful online instructors must structure social support if they 
are to maximize learning benefits for their students. Scaffolding is commonly used in building 
construction to provide support for the structure, add an element of safety to the project, and provide a 
secure place to stand for the workers. Similarly, communal scaffolding encourages and reinforces 
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cognitive development in the context of social connection and facilitation. As interpersonal dynamics are 
fitted into the existing course and institutional structures - through various online and offline strategies - 
learners are able to extend their range of learning opportunities by collaborating with others to achieve 
goals and complete assignments not otherwise possible. Finally, the scaffold helps instructors and others 
isolate individual student needs and customize communication to address a range of learning styles and 
socio-cultural variables. 

The idea of scaffolding has been used by others to describe the facilitation or transfer of knowledge from 
cognitive to practical applications in order to maintain optimal levels of challenge for students 
(Greenfield, 1984; Harley, 1993). In those instances it has been used to help visualize how the gap 
between task requirements and skill levels can be bridged. When we talk about communal scaffolding 
here, we are referring to bridging the gap of another kind - the gap between the task (cognitive, 
intellectual) and interpersonal (social, interpersonal) requirements of online learning. 

In light of the social dynamics found within effective learning environments, here are several online 
strategies students that we suggest to develop a solid communal scaffolding: 

❖ Personal Discussion Folders - We encourage you to begin your online experience by creating 
folders titled “Autobiographies,” “Introductions,” “Ice-breakers,” or “Name and Face,” where 
your e-personality can be posted. If possible, complete the passport effect by including a 
personal photograph. We’ve found that students report feeling closer or more connected to other 
students in the course when they can see their photographs. This is one of the reasons that we 
include our personal photographs as part of the faculty homepage. 

❖ Immediacy - To give your classmates the impression that you are “present” with them, respond 
to e-mail or threaded discussion in a timely manner. We also recommend using the other 
person’s first name in your reply to create greater interpersonal awareness. Even if you are just 
checking in to catch up on the day’s postings, stop by one or more of the rooms and let your 
presence be known in some way, shape or form. Consider it akin to simply “showing up” for an 
on-campus course. 

❖ Live Chat - If “virtual (live chat) office” hours are available through an online chat function, we 
recommend that you schedule an appointment at least one time a semester with your instructor. 
Students tell us that live chat of this kind helps them connect with us - at a safer distance than a 
phone call - early on in the semester in ways that e-mail or voicemail cannot. 

❖ Personalized E-mail - Another way to connect with faculty and other students is to send 
personalized e-mail outside of regular class time or required course discussion. Personalized e- 
mail might be used to encourage a fellow student who made a solid contribution in one of the 
required discussion formats. 

❖ Audio/Video: We’ve found that our tone of voice can be used to set the right mood for future 
communication. It becomes a percq)tual framework through which subsequent communication 
(whether textual or otherwise) is filtered. The tone of an audio message sent just before an exam 
or final paper to another student can even help reduce anxieties. All you need is a microphone 
and a plug-in such as RealProducer, which is a free download on the Web. If you’re really 
daring, you can use RealProducer or Microsoft Windows Movie Maker and add video. 

❖ Create Private Places - This might sound strange at first blush, but to the extent allowable by 
the instructor and course management platform, create a separate private area for you and your 
assigned discussion group apart from general class discussion. This is the same idea as the 
personal discussion folders mentioned earlier, but for individual groups only. This is a space 
that the instructor may not enter unless invited. It is a group “safe haven.” 
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❖ Regular Updates - Everyone likes to see fresh, new contwit, which is an important indication 
that value is constantly being added to your learning experience. As instructors, we smd weekly 
updates to students related to course content and procedures. Some students adopt a similar 
model and send brief weekly updates to us about their progress on group projects and theses. 

Group Discussion and Discursive Style - Last, but certainly not least, one of the most basic, but often 
most underestimated, online strategies you can use to build connectedness revolves around participation 
in required group discussion formats. If you’re not careful, your discursive style in online discussion may 
prevent you from connecting with others. While it’s valuable to critically challenge ideas, watch that you 
avoid accusatory language or leading questions that indicate your biases. Feeling “safe” to express one’s 
views is an important part of building community. Safety is further enhanced by establishing early on in 
the course rules for appropriate engagement and conduct within required discussion folders. 

Conclusion 

So, how do we encourage active engagement on the part of teachers and learners, and how do we 
contribute to the kind of communal infrastructure that makes learning fun and exciting? Perhaps when it 
is all said and done the final answer resides in some of the key indicators of community. For example, do 
you know each other’s names and are you comfortable engaging classmates and instructors in 
conversation? Does the conversation extend outside of the traditional instructional context? Do you share 
common goals and aspirations? Is there a sense that the “classroom” provides a safe environment for 
exploration and discovery? Are the learning outcomes ones that make sense to you and can you relate to 
them on a personal level? Have both students and teacher made an emotional commitment to the course? 

Is there a sense of shared responsibility? If most of these are present it is a good indication that your 
communal scaffold is a strong one and that you are well on your way to experiencing a rich learning 
community. It should be noted that there are no shortcuts to developing community. It takes time, and 
there is no substitute for time spent in communication with others — ^whether online or offline. Of course, 
time alone is insufficient. The time spent with classmates and with the instructor must be structured in 
such a way to facilitate the all-important transfer of intellectual and emotional capital. 
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Introduction 

The integration of supplementaiy interactive videoconferencing programming into existing curricula is 
providing K-12 educators and content providing organizations with greater opportunities for expanding 
their reach via distance learning technology. However, many teachers and content providing 
organizations are looking for distance learning applications that extend beyond “electronic field trips” and 
static, didactic interactions. Even though videoconferencing experiences have been growing in popularity 
(especially in the early grades), they are often single exposure interactions that lack the potential to truly 
eiu’ich the curriculum. Distance learning instructors are already beginning to think beyond traditional 
videoconferencing uses and are considering worthwhile partnerships and collaborations with other 
distance educators and content providers. They have realized that successful videoconferencing events 
are built on appropriate design and delivery strategies that are key to innovative distance learning 
programming. This paper will examine these notions and will highlight exemplary strategies that lead to 
highly interactive programs. 



The Impact of Distance Learning Technology 
The Interactive Experience Model 

Understanding the best methods for successful and innovative distance learning instruction first requires 
educators to examine how they currently approach their regular instruction. The Interactive Experience 
Model (lEM) detailed in Figure 1 provides a graphic representation of how the three contexts for learning 
form a foundation for an interactive learning experience. Although designed by Falk & Dierking (1992) 
to explain how learners interact in an informal learning environment, the lEM has implications for any 
classroom. The lEM is comprised of the Personal Context, the Social Context, and the Physical Context. 




Figure 1. Interactive experience model: from Falk, J.H. & Dierking, L.D. (1992). 
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The personal context takes into account individuals’ prior life experiences and expectations. Therefore, 
students with prior exposure to a particular topic will react and behave differently than novice learners. 
Additionally, students bring their own personal interests and understanding to the table, thereby 
influencing their interpretation of the information. As individuals come into contact with one another, 
they create the social context. This would include those interactions with family members, friends, peers, 
and teachers. Discussions that occur between learners further help the individual define the interpretation 
of experiences. The physical context is the actual setting, the physical environment. The classroom, 
chemistry lab, cafeteria, reading circle, and playground make up the physical environment as well as the 
materials that are held within those environments — desks, chalkboards, books, milk-cart, science 
equipment, maps, and so on. The physical context is the multi-sensory environment, those things that can 
be seen, heard, tasted, touched, and smelled. The combination of these three contexts provides the 
interactive experience. A learner takes into account prior life experiences, the influence of those around 
them, and their physical environment to generate understanding. 

The Learning Environment Via Distance Learning Technology 

While the lEM provides a framework for understanding how a learner is engaged in the regular education 
environment, the integration of distance learning technology can provide a unique quandary. Students 
involved in a videoconferencing experience, whether a single “electronic field trip” or semester-long 
“virtual course”, still interact within the three contexts. They still bring with them a personal context of 
agendas, background experiences, and learning modalities. They still interact among a social group, 
whether collaborating on a presentation with an on-site partner or answering a question posed by a remote 
site peer. However, the physical context becomes a divided entity (Figure 2). 



Full multi-sensory 




Auditory experience (limited by 
microphone & speaker system) 



Figure 2: The lEM with learners participating in a distance learning connection: 
from Barshinger, T. A. (1999). 

The technology has enabled the learners to experience and interact in two distinct physical contexts 
simultaneously. The students are now faced with not only interpreting the physical context in which their 
bodies are located, they must concurrently interpret the environment in which they are receiving a 
secondary source of auditory and visual stimulation. Therefore, learners must surmise a physical context 
while simultaneously experiencing two separate, yet connected, environments. 

The remote site environment is also limited in the types of auditory and visual experiences it can provide. 
Auditory technology such as microphones and speakers can impede the sounds and voices that are being 
delivered. Similarly, a learner can only see what the remote site camera can fit in its range. Low-end 
monitors can jeopardize the quality of visuals that are being transmitted. Additionally, sensory 
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competition from the home site can distract the learners from the content being delivered by the remote 
sites. Hallway noises, errant smells, and uncomfortable chairs can provide hurdles for attention. Further 
complications can arise simply from the technology itself. New users of the equipment may be more 
captivated by the workings of the technology than the content being delivered over it. The students’ 
technical orientation becomes even more apparent should a technical glitch, such as lost audio or 
streaking video, occur. Attention is immediately focused to the equipment. Therefore, a difficult 
question arises: 

How can a multi-sensory approach to learning be used to promote meaningful, relevant, and engaging 
experiences for students via a limited sensory technology? 

The answer to the question above is not an easy one. No distance learning connection will ever replace 
the experience of an actual on-site learning environment. However, engaging teaching strategies and 
unique production techniques can help overcome some of the limitations of the equipment. Before utilize 
these strategies, distance learning educators should reexamine their understanding of their role as a 
videoconferencing user. 



Going the Greatest Distance 



Thinking Beyond 

Entering the world of distance learning can be both ominous and exciting for educators. They are faced 
with unfamiliar equipment, new policies, different procedures, and that general feeling of uneasiness that 
often accompanies the thought of becoming a novice all over again. At the same time, teachers are 
presented with rare educational opportunities that may not have been available otherwise. When working 
with content providing organizations, such as a museum, zoo, or cultural agency, educators needs to think 
beyond the traditional modes of interaction and engage the content provider as a partner with whom they 
can collaborate, create, and instruct. Below are three su gg estions for thmking beyond the typical 
engagements with content providers. 

❖ Think beyond being just a recipient of information. How can you be an originator of 
information? The greatest feature of videoconferencing technology is its flexibility to 
synchronously deliver and receive information. Educators should ask themselves: “What 
content do I have to share?” and “How can I produce programming others would want to 
utilize?”. By answering these questions, educators may discover the role their own expertise 
can play for creating original content programming. 

❖ Think beyond “tts” and "them. ” How can you be part of “we”? The nature of 
videoconferencing technology sets the stage for an “us” (near end) and “them” (far end) 
situation. Different physical locations contribute significantly to this interpretation. Teachers 
need to remember the power of collaboration when engaged in instruction via distance learning 
technology. This collaboration could include a team of educators designing a standards-based 
program on pond life with the local science center, or a group of students organizing research 
for a debate on the topic of conservation for a nature center in another state. The efforts made 
to create materials and activities that stimulate meaning exchanges are necessary for making the 
technology medium “disappear” and the concept of “we”, as one group, viable. 

❖ Think beyond what a content provider can do for you. What can you do for them? While 
some novice distance learning educators may eventually grow into originators of information, a 
greater challenge can be in understanding what you can provide for the zoos, museums, and 
other organizations from which you request information. There are the obvious public relations 
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opportunities. Educator recommendations of quality programming to their colleagues increases 
a provides recognition and revenue. Just as important though, is the evaluative and community 
outreach resource educators become. Providers rely on feedback, obtained through evaluations 
and program requests, to assist in creating new events and revise existing ones. 

Exemplary Strategies for Distance Learning Content Delivery 

Raising the production level. While many educators may balk at the notion of bringing “Hollywood- 
like” techniques to the distance learning classroom, many simple production strategies can help raise the 
level of interest and quality for a distance learning event. These techniques are often overlooked as the 
average teacher or content provider educator lacks the knowledge for producing quality visual media. 

*1* Lighting. Bring in external lights to help brighten dark spots at the broadcast site. Adjust 
window shading and/or overhead lighting to darken over lit areas. 

❖ Sound. Wear a lapel microphone if available. Use multiple microphones wfien moving 
throughout a wide presentation space. Adjust sound mixers prior to a connection when using 
supplemental audio sources. Incorporate sound effects that can be delivered from a PC or other 
multi-media source. 

*> Camera placement. Avoid excessive backlighting by keeping the camera from pointing into a 
light source. Set presets for easy camera transitions. Use multiple cameras and video mixers 
for extra “wow” effects. 

*1* Scripting. A scripted outline that includes recommended dialogue, suggested questions, and a 
cross-reference of camera shots and video sources can assist new users and provide consistency 
between programs. 

Integrating pre-, post-, and program activities. Many educators and content providers have realized 
that single, isolated connections do little to increase students’ understanding of a topic. Introductory and 
follow-up materials and activities are necessary to help reinforce the content presented during distance 
learning events. Many content providing organizations provide lesson plans, supplemental resource 
guides, and on-line activities and games to help supplement their event content. 

Incorporating multi-media. Most videoconferencing hardware provides the flexibility to utilize 
supplemental video sources. In addition to VHS and DVD ports, PC centric systems and scan converter 
boxes for non-PC centric hardware can provide any user with access to a digital media source. 
Surprisingly, very few users take advantage of this capability. Some of the easiest ways to incorporate 
multi-media in a videoconference event include: 

❖ Pre-produced video segments from a VHS or DVD player. 

❖ Opening & closing “credit” sequences produced in Powerpoint, Director, or other similar 
applications. 

❖ Live video streams from related web resources. 
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Self-directed Executive Education courses are often used to teach managers and executives the skills they 
need to succeed in corporate leadership. The audience for these courses is typically the managers across a 
company or business unit within a corporation. Common course topics include leadership and 
management styles, strategies, and tactics, motivating employees, project management, business strategy, 
and many others. Executive education courses differ from traditional business academic courses in that 
they are generally shorter and do not offer academic credit for successful completion. We have created 
many executive education courses in our role as senior instructional designers for Option Six, Inc. In this 
presentation, we will describe one of the most successful course models we have developed and 
implemented. First, we will explain the pedagogical approach we have used. Next, we will describe 
relevant course features and discuss the instructional design challenges we have encountered. Finally, we 
will demonstrate the model by showcasing a recently completed course project. 

Starting With Values 

At Option Sbc, we’ve adopted a values-based approach to developing e-leaming courses and other 
instructional materials. “Values-based” means starting out by identifying and describing our fundamental 
values about learning, values which frame the highest-level design context from which we develop 
specific instruction. Starting with values further means that all subsequent design decisions must support 
these values. When choosing between two or more generally equal design paths, the path that better 
supports these core instructional values is the preferred path to follow. 

Just knowing our values isn’t enough, of course. Once we’ve established our values, we are able to 
identify specific learning goals that are aligned with and support these values. Once learning goals are 
specified, alternative instructional methods are considered in order to create the most appropriate learning 
experiaice for the target learners. Specific contextual conditions resident in every development project 
(such as, audience characteristics, corporate culture, and client constraints) influence the selection of the 
best-fitting instructional methods in each case. By considering the situationalities (values, outcomes, and 
conditions) present in the learning environment as we choose specific instructional methods, we create ' 
instruction that is more likely to be effective than that designed without concern for the situationalities 
(Reigeluth, 1999; Beatty, 2002). 

Fundamental Values 

We’ve identified sbc interrelated fundamental values that all Option Sbc courses must uphold; leamer- 
centeredness, flexibility, context driven, effective, solutions centered, and interactive. Some of these 
values overlap with each other, but each is important in its own right. (In our presentation, we’ll discuss 
each of these in detail.) 
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The Importance of Engagement 

In our experience, both online and in the classroom, students who are “engaged” in the learning process 
perform better on post-learning assessments — th^ learn better. Two important factors in engagement are 
motivation and interactivity (Kearsley & Schneiderman, 1998). 

The single most important factor in learning is student motivation. Students who are not motivated are 
never engaged. Unmotivated students rarely transfer new knowledge into the work place. On the other 
hand, motivated students put forth extra effort to learn, follow the guidance of the course, and try to use 
all the learning resources provided in the course. In short, all successful students are motivated students. 
One of the most useful ways to think about student motivation is the Keller’s ARCS model: Attention, 
Relevance, Confidence, and Satisfaction (Keller, 1 979). Interactivity is not well-defined in the e-leaming 
field (Rose, 1999; Sims, 1999). To us, interactivity refers to the student interacting with the course 
material at an intellectual level. Simply requiring a student to click to proceed to the next screen is not 
interactive, in our use of the term. Rather, challenging the student to consider a new concept, apply a 
concept to a new situation, and engage in a learning-focused activity at the same time (search for 
information, fill out a form, post a thought to a discussion, etc.), meets our criteria for interaction. 

Theory Meets Practice 

All of our online courses follow a similar pedagogical approach, custom-fit to specific content and clients 
as needed. In general, we believe in the supremacy of learning by doing as an approach to learning that 
can be effective with all students in every learning situation. It is our goal to challenge each student to 
perform - using relevant skills and abilities that apply newly acquired knowledge. Option Sbc courses 
implement four major instructional strategies - presentation, practice, assessment, and performance 
support. 

Presentation 

In all purposive instruction, there must be a method of content presentation, if the learner is expected to 
gain new knowledge, skills, and ability. In Option Sbc courses, presentation takes many forms, including 
text descriptions, diagrams, pictures, video explanations, multimedia demonstrations, audio clips (of 
information or examples), and on and on. 

Embed objectives. It has been widely accepted that presenting a list of learning objectives to students at 
the beginning of a presentation helps them structure their learning efforts (Mager, 1984). In practice, 
however, we have realized that most students do not even read lists of objectives, let alone plan their 
learning around them. We have found that a more effective practice is to embed these objectives in the 
introductory “welcome” to the course. All students read this section. Of course, the instructional designer 
must use a set of formal learning objectives when designing the instructional experience, and these 
objectives should be available to the student who wants to see them, but th^ should not be used as the 
Gateway to learning. 

Use multiple presentation media. Presenting complex concepts should be done in multiple modes. 
Simply writing 2-3 paragraphs of concisely worded, and technically accurate text is usually not enough. 
Often, we supplement text with a multimedia explanation, video demonstration, detailed graphic 
explanation, or audio explanation. In all cases, a text explanation should be available for students, since it 
is usually easy to print and read later and is also accessible to screen-reading software when needed. 
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Summarize key information. Occasionally throughout a presentation section, and especially near the 
end of one, it is important to summarize the key points presented to help the student remember what is 
“really” important. Presentation sections should be kept short enough that a student can easily remember 
3-5 k^ points covered in that section, and so the student can synthesize several concepts learned together 
as the student completes practice activities. In the performance support resource section, the k^ points 
from each major topic should be provided for student use outside of the course environment. 

Practice 

Practice opportunities are essential to effective learning. Option Six courses provide many opportunities 
for student to practice applying the knowledge and skills they are acquiring, as they are learning them. 

Learn by doing. Practice items challenge students to continue to learn as they attempt to “do” something 
with the knowledge they are gaining. Simulations, role-play scenarios, and other inunersive activities are 
often excellent choices for practice (and assessment). These activities should be motivating, meaningful, 
allow for learning through mistakes, and provide robust coaching and feedback. Not every course topic 
needs an extensive simulation for adequate practice, of course. Sometimes, completing a simple 
spreadsheet or reading a short case with several decision-making questions is enough. 

Support Students should be supported (or guided) as they practice, though only as much as each 
individual student needs. In the beginning of a course, students may be expected to need a high level of 
guidance, though a few may not. Later in the course, many students may not need much support at all, 
though a few may still need a high level of support. It is important to try to meet the needs of both types 
of students in both practice situations. 

Feedback. Feedback can come from many different sources, including the course system, an external 
mentor or instructor, or the student her/himself Feedback can also be immediate or delayed. Providing 
immediate feedback is especially important when later practice builds upon the results of previous 
practice. 

Progression. In general, practice opportunities should progress from simple to more complex, from easy 
to difficult, and from narrow to wide scope. Following a standard flow such as this helps students choose 
which practice items they would like to attempt, especially in later topics 

Assessment 

Assessment in Option Sk courses is implemented in several ways. First, most courses use a 
comprehensive pre-assessment to help the student understand what they already know (and don’t know) 
about the course topics before they start learning. Throughout the presentation phase of each course topic, 
small, embedded self-assessments help the student understand how well s/he is learning the course 
information, so that s/he knows whether to proceed with the next section of presentation, or to go back 
and review the current section. Practice opportunities (described above) provide another opportunity for 
assessment, as the student “tries out” new knowledge and skills, receiving feedback about their 
performance. Finally, most courses include a comprehensive post-assessment which helps the student 
understand which course topics they have learned well, and which topics they may need to review again. 

Performance Support 

To be truly effective, executive education courses should also be useful as performance support tools. 
This means that students must be able to use course material to aid them in the performance of their job 
duties after they have completed the course. One mechanism that facilitates this use is the course (or 
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topic) “key points takeaway,” a resource that summarizes the key learning points for a course (or topic) 
and provides other helpful hints or guidelines for them to implement outside of the course environment. 
Another helpful course feature is the ability to search and access course material by topic and sub-topic. 
When a student finishes a course, they should be able to return to the course to refresh their knowledge 
about a specific course topic or to retrieve a learning resource s/he may now need on the job. Another 
mechanism that is used for performance support is “downloadable tools” that the student uses during the 
course, but that are also useful on the job. These take the form of Excel spreadsheets. Word templates, job 
checklists, financial calculators, and more. 

Implications for Design 

We have developed a common interface that supports student learning in the self-paced, individualized 
online setting we have described above. Since we typically have a range of clients for a particular course, 
we leave room for customization to meet specific client needs (technical systems, collaboration 
requirements, etc.). 

Course Sections 

Even though we allow for customization, all of our courses support our fundamental values about 
learning, and most of our courses include the specific sections described below. 

The Introduction/Welcome section introduces the course topic(s), explains overall learning objectives, 
and begins to engage the learner in the topic and context. The Content presentation section presents 
information about the course topics in multiple media, including text, animation, graphics, video, etc., and 
provides links to extra information (marginalia) and external websites. It includes short, embedded self 
assessments and instructive practice activities (guided practice). The Resource center section provides 
access to downloadable tools (if any), “Key Points” topical summaries, the course glossary, lists of other 
websites or readings that might be helpful, and an annotated bibliography of other useful resources. The 
Practice arena section provides access to all the practice activities for a topic (or course). These practice 
activities are also accessible from the content presentation section. Co-locating all the practice for a 
course in one place allows a student to re-try a practice activity from a previous topic, or attempt a 
practice activity from a new topic, without having to search through the content sections. Finally, the 
Assessment section provides links to the pre- and post-assessments. Students use this section to check 
their knowledge, reviewing content and revising their learning path (if desired) at the same time. 

Evaluation and Conclusion 

During the presentation, we will demonstrate the design features with an existing course, and present 
findings from our evaluation of the latest version of this course model. (This evaluation is in progress at 
the time this is being written.) 
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Introduction 

The distance education literature reports many success factors common to managing the development of 
technology-based university-level courses. Commonly cited success factors include effective use of 
changing technology, acquiring sufficient resources for development and operation of distance courses, 
creating effective course designs, effective staffmg, conducting sufficient planning, accommodating the 
needs and characteristics of instructors and students, and acquiring sufficient technical expertise. Much 
of the existing literature focuses on technology-based traditional university training. In contrast wdth 
traditional university programs, technology-based professional training and certification programs face 
unique challenges, such as the requirement that sponsoring organizations pay student salaries while the 
students use the learning technologies. Also, the training investment is expected to transfer duwtly into 
job performance. Thus, incentives to create effective learning environments are very strong in 
professional programs. 

Defense Acquisition University (DAU) provides professional education for the U.S. Department of 
Defense Systems Acquisition Workforce. More than 15 technology-based courses have been developed 
by DAU. Collectively, over 20,000 persons graduate annually from these courses. Annual graduation 
rates for each course range from several hundred to nearly 10,000 students. This study investigated 
success factors for managing the development of technology-based courses. The research team attempted 
to identify success factors that might be relevant to DAU courses. In order to pursue this research goal, a 
literature review of success factors was conducted, followed by a study of eight successful DAU 
technology-based courses. The eight courses studied were considered successful based on favorable 
student and instructor feedback, meeting or exceeding projected enrollment numbers, and high levels of 
manag^nent and stakeholder support. The courses studied range from lower level courses taken online 
with no required instructor interaction to higher level courses using sophisticated threaded storylines and 
hybrid (online and classroom) compon«its. One course uses conference calls and an online forum tool to 
create a “virtual classroom,” combining the convenience of “anywhere” with real-time instructor-student 
and student-student interactions. 

The researchers used success factors described in the literature to help guide the data collection. The 
lito^ture provided a rich background for the study of the successful DAU courses. After the literature 
review, interviews were conducted with the DAU course directors, followed by analysis of data. The 
success factors identified in the literature review were then compared with success factors identified by 
DAU course managers. The study provided a strong grouping of success factors that should apply to 
DAU as well as to the management of other government and traditional higher-education distance 
learning programs. 
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Review of Literature 



To simplify review of the immense body of literature related to distance education, the selection criteria 
for the literature search was very narrow, focusing on reports of success factors for the management of 
distance education development projects. The search favored empirical results from controlled studies 
where possible. Additional literature that reflected collection of data from experienced distance educators 
was also included. All items reviewed had some discussion related to success factors for the management 
of distance education development programs. After a preliminary review, several of the items were 
reviewed in-depth, leading to an identification of general categories along with some specific success 
factors (Alexander et al., 1998; Bates, 2000; Brigham, 1992; Lopez & Nagelhout, 1995; Volery, 2001; 
Wagner, 1995). 

Robinson’s (2001) summary of common problems related to innovation with on-line distance learning 
provided a useful categorization of success factors. Based on the experience of 426 distance educators, 
Robinson classified distance education course issues related to innovation, leading to the four general 
categories of resource availability, organizational issues, human resource capacity^ and technology 
capabilities. These four categories of innovation problems comprised a convenient organizer for the 
success factors found in the literature. Table 1 lists the success factors identified in the literature review, 
sorted according Robinson’s categories (2001). 



Table 1 : Literature Success Factors by Category 



CATEGORY 


SUCCESS FACTORS 


Resource Availability 


Sufficient fundamentals; time, funding and personnel 
Proper infrastructure for technology support 
Use of deadlines to help manage resources 
Reasonable project scope 


Organizational Issues 


Involvement of appropriate organizational levels 
Appropriate organization for technical support 
Adequate coordination and administration practices 

Organization-wide strategies for the use of technology in teaching and learning 
Organizational support for learners and instructors 
Attitudes toward use of technology 


Human Resource 
Capacity 


Produce and support learning technology 

Apply systematic and analytical methods 

Think through and plan the details required for distance learning 

Ability to address specific student needs 

Use the technology to enhance learning in new ways 

Use a sound and well-integrated instructional strategy 

Provide learner support 

Design assessments appropriate for technology-based delivery 
Staff training/development in technology where needed 


Technology 

Capabilities 


Adequate technical support 

Software testing expertise available 

Software development expertise (where relevant) 

Copyright issue resolution 

Adequate / equitable student access to hardware and software 
Ability to absorb extra technology-related costs 



Data Collection Method 

The research design was largely qualitative, with a guided interview as the primary means of data 
collection, followed by data analysis and a correlation of findings with the success factors identified in the 
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literature review. The interview protocol was designed to facilitate exploration of the course managers’ 
experiences with general issues identified in the literature review. Thus, the literature review was used to 
help construct the data collection protocol. The protocol questions were designed to provide field-based 
inputs from the eight DAU developmental course managers, facilitating a comparison of their experience 
with the success factors identified in the literature review. The courses covered a range of subjects 
including management of acquisition programs, funds management, management of system testing, and 
software systems acquisition. The interview method was face-to-face sessions with some follow-on 
contact for elaboration where needed. A standard qualitative review process was followed for the 
analysis of data including review of interview transcripts with identification of key points, organization of 
key points into the main themes, selection of the most frequent themes as the candidate success factors, 
and review of the findings by a course manager. A post-hoc analysis was conducted to correlate the 
success factor data with the results of the literature review. 

Results and Findings 

The interview protocol was tested with an initial interview. All interview sessions were managed to 
make sure that each protocol question was addressed. The interviews were recorded and transcribed for 
analysis. The analysis of data included a review of the transcripts for identification of key points made by 
the course managers. The initial review produced 99 independent candidate success factors which were 
grouped in categories according to logical themes. The final theme categories were management issues, 
development process issues, human resource issues, course design issues, and technology issues. Next, 
an additional layer of themes was identified, creating sub-categories within the categories. The top 10 
sub-category themes were selected as candidate success factors based on high levels of occurrence. These 
top 10 themes are not inclusive of all of the 99 issues raised, but they were the most recurrent themes 
across the DAU courses. The final list of success factors emerging from the data, organized by the final 
theme categories studied, is illustrated in Table 2. 



Table 2: DAU Success Factors by Category 



CATEGORY 


DAU SUCCESS FACTORS 


Management Issues 


Project Planning and Management Techniques 
Integrated Development Team 


Development Process 
Issues 


EfiFective Blending of Technology Alternatives (also a technology issue) 
Effective use of Testing and Evaluation 


Human Resource Issues 


StafiBng and Team Issues 
Availability of SME time 


Course Design Issues 


Selecting a Motivating Instructional Strategy 
Consideration of learning variables 


Technology Issues 


Configuration Control (also a development process issues) 
Long-term technology support 



These 1 0 DAU success factors were then reviewed and rated by importance to each DAU course included 
in the study, based on the data collected. A rating of 1-5 was assigned to each success factor for each 
course in the study. The ratings were tabulated, and the success factors were then ordered by importance. 
The following success factor list is sorted in tabulated order of importance to the DAU courses. A 
description of each success factor is also included in the list: 

1 . Configuration Control - This includes document control, harmonizing design and development 
versions, ensuring that source documentation exists for all on-line materials. All course managers 
had to come up with a way to successfully manage configuration control issues. 
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2. Effective Blending of Technology Alternatives - This includes analysis of available technologies, use 
of the most efficient mix of technologies, classroom sessions, on-line courseware, and others, and 
consideration of future changes driven by new technology. 

3. Staffing and Team issues - This includes the optimal team composition, a positive and supportive 
work culture, protecting the team from distractions, co-location of team members where possible, 
staff selection issues, and careful selection of working team member combinations. 

4. Availability of Subject Matter Expert (SME) Time - In several cases the classroom culture of the 
faculty made it difficult to get dedicated SME support for online course development, but all course 
managers were able to eventually get the necessary support. 

5. Project Planning and Management Techniques - This involves defining requirements for learning 
objectives before finalizing the course design, making management tradeoffs to achieve the optimum 
balance between quality and schedule, and using baseline measures to track course performance. 

6. Effective use of Testing and Evaluation — This includes early usability testing, prototype 
demonstrations for organizational stakeholders, formative testing during design, and testing by both 
instructors and students. 

7. Integrated Development Team - This includes building an integrated design and development team 
with a functional group review and decision-making process, as well as early inputs from all team 
members on critical design decisions. 

8. Selecting a motivating instructional strategy - This is particularly critical for courseware and online 
interactions. Effective strategies used at DAU include the use of problem-based and scenario-based 
training mechanisms, including the use of elaborate stories as an instructional vehicle. 

9. Long-term technology support- This includes long-term technology planning, consideration of future 
requirements for interoperability, ease of maintenance, and compatibility of courseware with future 
releases of browser “plug-ins”. 

10. Consideration of learning variables - This includes many factors such as planning adequate student 
time for course completion, matching the course level with the students’ level of preparedness, and 
providing efficient and usable resources to the students. 

Discussion and Conclusions 

The goal of the study was to identify success factors that might be relevant to DAU courses. The 
literature review produced an empirically derived group of success factors that guided development of the 
data collection protocol. Interviews with the eight DAU course managers confirmed that success factors 
in the literature were generally involved during their course developments. The interviews also provided 
an independent group of DAU success factors. Both the literature and the DAU success factors address 
issues related to human resources and technology. The success factors from the literature review focused 
on organizational issues, while the DAU success factors addressed more general project management 
issues. The DAU list added several success factors that were important to the DAU course managers, but 
were not emphasized in the literature reviewed. These included configuration control, availability of 
SME time, the use of an integrated development team, and the blending of strategies and technologies. 
Taken together, the combination of the literature review and the DAU success factors are a reasonable 
group of issues for both new and existing managers of technology-based development programs to 
consider. Some of these success factors may be more applicable to specific institutions, and the interview 
data from the study suggest that the professional workforce training focus of the DAU environment may 
be responsible for the identification of the additional success factors. Future research should elaborate on 
the role of these additional success factors and clarify mechanisms for their application in ongoing 
distance education development projects. 
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Substituting for Yourself: 

The Art of Being in Two Places at Once 

Marilyn Bergmann 
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College of Business 
University of Wisconsin-Eau Claire 

Thomas Bergmann 
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Just this month, our faculty teaching online received an email announcing a very important workshop that 
we had scheduled with our consortium partners. These same online professors teach courses on our 
campus and the workshop coincided with the end of the first week of summer session. At least two of the 
faculty replied immediately that they could not attend because they would be teaching. Neither of them 
had thought of the possibility of using Blackboard in their absence although both are well versed in its 
use-after all, they teach online! In fact, one of them uses Blackboard regularly to support her campus 
classes. 

Faculty who are active in their profession often have to be away from campus at least once a semester to 
attend conferences and engage in other professional activities. The traditional approach when faculty are 
gone is to cancel the class, have a guest speaker, give an examination, give an additional assignment, or 
have a colleague take the class. Most professors are really not satisfied with any of Aese approaches, but 
believe they have few alternatives. Canceling class is usually fi-owned upon by administrators and 
students discount the information received from guest speakers and faculty fill-ins. They respond by 
cutting the class and viewing additional assignments as busy work (their performance on these 
assignments often shows this perception). 



WBLS as a Solution 

Why is Blackboard or any other Web-based learning system (WBLS) a viable solution to those days you 
just can’t be physically present? 

❖ It guarantees that there’s no discrepancy between what you think is covered in the class and 
what is actually covered. No two individuals teach exactly the same. They don’t use the same 
examples. They don’t give equal emphasis to ideas. Thereftjre, even if you have called in a 
favor from the best instructor on campus and carefully outlined the topic of the day, test 
questions based on that material are best avoided since students will inevitably respond, “that 
wasn’t covered.” But, if you have created the material, placed it in Blackboard, and observed 
and interacted with the students as they use that material, you know for a feet that “it WAS 
covered.” In addition, with this approach, you don’t have to bum up your goodwill among 
colleagues by calling in fevors. 

❖ It guarantees that the class progresses as plarmed. Who hasn’t had an occasion when you 
absolutely have to miss a class and you’ve pulled out the videotape that is “somewhat ’ pertinent 
or scheduled that exam from nowhere. Those solutions fill the class time, but they are usually 
not the same instruction the students would have received had you been in your usual place at 
your usual time. Most professors distribute a relatively detailed syllabus the first day of class 
outlining what will be covered and a tentative time line. The scheduling of materials is usually 
done to optimize the learning environment for the student. Cases and other activities are 
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likewise scheduled to maximize the leaning outcome for the students. When you must be gone 
and the schedules are adjusted or an assignment moved or substituted, it can disrupt the flow of 
the material. This happens when the professor substituting may not be comfortable with the 
material scheduled for that day and thus a topic is moved forward that he/she would prefer to 
lecture on for the class session. But, with Blackboard, you are virtually present and can utilize a 
time span rather than a particular time so that the class is right where you want it when you 
return. The material you want included is covered when you want it covered, and any activity 
you want to have integrated with the material can be. 

❖ It guarantees that your students are involved and active during your absence. The substitute 
whose material won’t be included in the test (students are pretty savvy about this) or the student 
showing the videotape that isn’t quite pertinent is usually asked to pass around a sign-up sheet 
and verify that the number of bodies in the room matches the number of names on the list or 
human nature may prevail. Some faculty members do not tell students ahead of time if they are 
going to be gone because the absenteeism rate will be too high. But, your own material in 
Blackboard with student interaction that you can “tune in on” and participate in from wherevCT 
you may be is much more likely to assure your students’ “presence” during your absence. By 
reviewing the interactions around the discussion issue you can determine the frequency of each 
person’s input and the quality of their comments. 

For feculty who have had little or no experience with WEBLS products like Blackboard, using it may 
appear as a daunting task. However, with a minimal amount of help from an instructional designer, 
converting course material is a relatively easy task. Instructors can take material for a class period or 
multiple class periods and convert it for the virtual classroom so that it becomes a seamless part of the 
class. The direct results include: 1) students receive the usual course content presented in a way chosen 
by the instructor, and 2) the instructor, as long as there is internet access, is literally in two places at once 
and can monitor and take part in the interaction that occurs within the class. 

A Basic “How To” 

The first step is to decide what materials you want the Web-based learning tool to provide to your 
students. It can be a lecture outline, a case, a PowCTPoint presentation, links to WEB pages with 
directions for what to look for on those pages, links to online articles—Just about anything you can 
imagine. Again, if you’ve used a tool such as Blackboard before, the process of putting the materials into 
the tool is the easiest part. If you have not used such tools, this is probably not the time to try to learn it 
unless you are planning your absence far in advance. Rely on your campus LTEKD (Learning Technology 
Development Caiter) or similar unit to help you place the materials into the tool. Concentrate, instead, on 
how to use the tool in a meaningful way. The good news is that functioning within Blackboard or any 
WBLS is quite simple--especially if you are comfortable maneuvering on the WEB. Students are almost 
always either already familiar with the tools or quick learners. And online tutorials and manuals 
accompany the tools. In addition, your LTDC will likely have a quick and easy handout or online site 
that coincides with whatever tool your campus has available. 

The next step is to determine how to make the WEBLS experience equivalent to the face- to-face 
classroom experience. Here are some essential things to remember: 

❖ Think proportion. If you would normally have your class read one chapter and one article in 
preparation for class, don’t assign five WEB readings. If your class is normally an hour long, 
don’t require “discussion” that will take your students two hours to complete. Your students 
should spend about the same amount of time in the virtual classroom as they would preparing 
for and participating in the traditional classroom. 
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♦♦♦ Provide a natural flow of material between the literal and the virtual classroom. The articles or 
cases students read online, the WEB-sites they visit, the lecture notes that they study must fit 
naturally with what has been going on in the classroom. Students respond no better to add-ons 
and fillers than they do to substitute instructors. If they sense that the online experience is there 
just to fill time, they will respond accordingly. 

❖ Introduce the virtual experience within the traditional experience. The students need to be told 
upfront what they will be covering online and what the expectations are. Since you are not 
there when they may be online, it is important to be overly prescriptive on what you want them 
to do and the level of performance expected. Within the traditional classroom, it is easier to 
clarify and provide immediate feedback to guide the learning. In the virtual environment, you 
need to carefully design your learning component prior to the session beginning. Unless you 
have highly self-motivated students, knowing they will “get credit” for participation in the 
online experience is helpful. 

❖ Supply clear technical directives. This often entails a very brief handout (how do I reach the 
virtual class) and clear paths throughout the platform to direct the student from activity to 
activity. Here is a place where the LTDC can help if you are new to WEBLS. 

❖ Require interactivity. Students who visit the virtual classroom once are much more likely to be 
superficial than those who must revisit the classroom, building on what they did in the first visit. 
So, minimally, set up a discussion where the students must make an entry and then return to 
read other entries and provide a response. The amount of time the students will be using the 
virtual classroom will impact the amount of intwaction. It may be helpful to provide them with 
examples of what you would consider a superficial response. For example, ‘“Cool idea’ or 
‘nice job’ does not add to the growth of knowledge.” 

❖ Maintain a presence. No matter where you are, the Internet is likely there. Find a way to log on 
and check in. This might include comments on how the discussions are going or answers to 
posted student questions or even messages from afar (digital pictures). 

❖ Provide closure. Again, for a full integration, the class that follows the WEB “classes” must 
flow fi'om them, building on wftat the students learned and did in the WEB-based platform. 

Examples of Success 

In the case of our institution, a number of professors have taken advantage of Blackboard in this way. In 
fact, we found Blackboard indispensable A^en one of the faculty, because of his administrative position, 
had to attend an AACSB (The Association to Advance Collegiate Schools of Business) meeting out of 
state. He was teaching a one credit graduate elective course on discipline that met for only five weeks; he 
would be gone the third week. He informed the class the first night that weeks 1,2,4, and 5 would be 
traditional and week 3 would be virtual. In week 2, the class led up to the point where the online class 
would pick up. A technology support person came to class for the last 1 5 minutes that night and 
acquainted the students with online maneuvCTing by walking them through the online documents (a brief 
handout was also provided). As the technician projected a computer image showing each documait on 
the screen, the professor briefly explained what would be required of the students. The activities 
included: 



»J» Completing a 10 question True/False questionnaire 
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❖ Viewing a scale that, based on their answers to the questionnaire, identified what type of 
disciplinarian each student would be in a work environment 

❖ Reading a “lecture” that was generously laced with charts and images 

❖ Reading 3 cases 

*** Visiting the discussion area to answer brief questions about the cases 

❖ Revisiting the discussion area and reading the answers provided by 3 other students for each 
case; thoi, providing feedback to those students that either expanded their answers, supported 
their answers with examples or detail, or offered alternative ideas (No “nice job” responses 
would count!). 

<* Taking a brief quiz that applied the lecture materials 

Week 3, the professor picked up with a discussion of the students’ findings regarding disciplinary types 
and provided a follow-up to their case results. He had been able to monitor their answers and their 
responses and make online comments during the time he was gone. 

A similar procedure was used for an undergraduate Business Telecommunications class. In this case, 
however, a single 50-minute class was all that the instructor would miss, so the online work involved was 
less. The students in the class wra^e already familiar with Blackboard from other classes (these were MIS 
students). They were simply told when the material would be available. The students were continuing a 
detailed case study project already begun in class. Previously they had developed a strategic plan. They 
were now at the implementation stage. Blackboard contained a document with new, essential information 
and a summary document of the previously covered relevant information. In the scenario, students were 
told th^ were to imagine they would be attending a meeting in which they would recommend the 
purchase of new equipment~the equipment would be expensive but the firm could afford it and it would 
be essential to their implementation. They were to outline the presentation they would make at the 
meeting and post it to the discussion area. They were then to return to the discussion area the next day 
and provide feedback to the two students whose outlines were posted below their own. The instructor 
provided feedback after the second postings. 



Conclusions 

There is no question that the “Be your own substitute via WEBLS” approach is more time-consuming 
than grabbing that video or calling in a favor and there are times when the video and the guest/substhute 
are absolutely appropriate. But the WEBLS approach is a viable alternative. And, it has some very 
positive features: 

1 . It is an advantage to both faculty and students because it allows faculty to use class time in the way 
th^ really want to use it. 

2. It makes faculty more willing and able to accept projects that take them occasionally away from their 
classroom. 

3. It stimulates interest and use of WEB-based Learning Systems by instructors. 

4. It acquaints students with the full capabilities of WEB-based Learning systems, making them ready 
for the WEB-based courses which are likely to be part of their future. 

5. It begins a “library” of materials for the faculty that become a ready resource to reuse and to build 
upon. 

So, next time that conference beacons or that seminar is really too significant to pass up, or the Chancellor 
requests the honor of your presence, consider being your own substitute. You really CAN be in two 
places at once. 
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Online Learning in K— 12 Schools: “What Woriis” 
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North Central Regional Educational Laboratory 

Since 1996, enormous progress has been made toward achieving the U.S. Department of Education’s goal 
to build a national technology infrastructure to support its vision for effective technology use in the 
nation’s elementary and secondary schools. Significant increases in federal, state, local, and private 
investment in the national technology infrastructure have helped many teachers and students in 
elementary and secondary schools obtain access to and begin using a variety of powerful new online 
learning tools. These tools enable distance learning and technology-based instructional delivery systems. 

Recent growth in national attention directed toward online courses and virtual schools underscores the 
importance of e- learning policy and online learning practices. The full complexity and impact of e- 
leaming and online courses on policy and practice in K-12 schools and school districts is emerging only 
now as a subject for consideration and discussion by leadership in public education. It is imperative that 
state-level education policymakers become active participants in the ongoing conversations about K-12 
online learning. Such participation will help ensure the systematic implementation of effective e-leaming 
strategies in the nation’s elementary and secondary schools. 

Policy and Practice 

At the CiTE Virtual High School Symposium, sponsored by the Center for Internet Technology in 
Education and held in October 2001, there was ample evidence that a vigorous and growing community 
of practice is building what could be called a virtual high school movement. 

In his keynote address at this symposium, John Bailey, director of the Office of Educational Technology 
for the U.S. Department of Education, shared his views on education and technology and described the 
conditions required to achieve positive outcomes from e-leaming in K-12 schools. He mentioned the 
following policy themes for online learning: 

❖ “Anywhere, anytime” learning means that “education can now be delivered to students 
wherever they are located.” 

❖ Online learning should encourage schools to become “education centers” for their respective 
communities. 

“Every educational program is a technology opportunity, and every technology program is an 
educational opportunity.” 

❖ Online assessment in conjunction with online learning has the potential to significantly increase 
the effective use of insfructional time and encourage “a system of education that isn’t based on 
mass production, but is instead based on mass customization.” 

❖ “We need to be relentless in measuring and assessing the impact that technology has on 
education and on academic achievement. We need evidence that teaching and learning are 
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improved as the result of technology. Using technology to teach using traditimial methods will 
only lead to traditional resuhs.” (Bailey, 2001) 

Dialogues and conversations among attendees at the symposium centered around creating a consensus 
between practitioners and commercial developers of online learning products about the critical issues 
facing the online learning community. It was not apparent, however, that state-level policymakers or 
representatives from education agencies in the states developing virtual schools took part in these 
important discussions. 

The National Association of State Boards of Education (2001 ) concurs that thwe is a lack of input from 
education leaders and policymakers with regard to e-leaming development and implementation: “The 
uncomfortable reality is that education leaders are not currently driving the policy agenda” (p. 6). Without 
the substantial participation of state-level educational leadership, howevCT, any possible contribution of 
learning technologies or e-leaming to school improvement and reform may fall prQ^ to the loose coupling 
that is apparent between many traditional state educational policies and visionary e-leaming practice. 

Findings and Recommendations 

In the end, student access to online courses will be determined by local decisions that must be made by 
education administrators and policy leaders everywh^e. These decisions will affect whether or not 
specific virtual courses will be approved (or afforded) fw individual students who have particular 
rationales and reasons fw requesting enrollment in online classes. In some cases, this situation may 
contribute to lower enrollment in regularly offered on-site classes, lower daily attendance, and shrinking 
instmctional loads — ^with negative impacts for staffing and personnel budgets. 

The following findings and recommendations offer some next steps for state and local policymakers. 

Finding 1 

Innovative leaders in the e-leaming movement and established state education policy leaders have not 
established a basis for communication and dialogue on critical policy issues. 

Recommendations 

Leaders in the e-leaming movement and state education policymakers should initiate 
communications and begin working together to help shape e-leaming practice. Such dialogue 
will help education policy leaders understand the unique dimensions of e-leaming practice and 
also will enable e-leaming to operate within critical educational policy cOTStraints. 

Key state education agency representatives should be encouraged to participate in professional 
fomms such as subsequent CiTE Virtual High School Symposiums. These fomms provide 
common ground for critical dialogues between e-leaming consumers, producers, and 
educational policy leaders concerned with the top-to-bottom articulation of e-leaming policy 
and practice. 

❖ State education agencies should be strongly encouraged to begin a thorough analysis of existing 
state education policies that have clear implications for support and regulation of online 
learning or e-leaming in K-12 environments. In all cases, due consideration should be given to 
modifying or adapting existing policies to promote the equitable diffusion and implementation 
of online learning. The time-consuming development of new policies, in contrast, may delay or 
slow the adoption and effective implementation of K-12 online learning. 
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Finding 2 



When provided with quality professional development opportunities and supervised online clinical 
experience, good traditional teachers also can become effective facilitators of online learning. Similarly, 
well-qualified and experienced online instructors can learn the more specialized instructional design and 
implementation skills that are necessary to create quality online learning materials based on their existing 
teaching experience and curricular expertise. 

Recommendatioiis 

❖ Certified, experienced teachers who wish to become online instructors should be required to 
complete an approved professional development curriculum ensuring their competency as 
online instructors before being assigned responsibility for leadership in an online course. 

❖ Experienced online instructors should be required to complete appropriate specialized 
professional development concerned with the design and implementation of online learning 
environments before undertaking the local development of online courses. 

❖ Specialized professional development programs that provide teachers with professionally 
recognized credentialing as online instructors or developers of online learning often are costly 
and time consuming. Professional development costs for teachers should be shared or fully 
reimbursed, work release time should be provided to support preparation required before and 
during initial online instructional assignments, and teaching loads should be appropriately 
adjusted to compensate for online teaching or participation in online instructional development 
projects. 

❖ School districts should avoid encouraging or requiring teachers to accept assignments as online 
instructors. Th^ should not require teachers and other district employees to participate in the 
development of online instructional materials or course materials without appropriate financial 
compensation for assigned duties and due respect for copyright and ownership of intellectual 
property (see American Association of University Professors, n.d.). 



Finding 3 

Hybrid courses (combining face-to-face and online instruction) with smaller enrollments and clear 
linkages to approved curriculum practice seem to offer higher completion rates and arguably better 
quality learning outcomes than online courses alone (Cavanaugh, 2001). This finding suggests that online 
learning optimally should be used in some ccanbination with face-to-face instruction, primarily from 
qualified and experienced teachers ^^o are in physical proximity to enrolled online students. 

Recommendation 

❖ School districts, state education agencies, and the U.S. Department of Education should 
collaborate on development and implementation of a scientific research agenda related to the 
use of online professional development and e- learning with students in K-12 learning 
environments. This agenda should determine ^^ich resource configurations and instructional 
design practices optimize student achievement and authentic learning outcomes. 
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Finding 4 



Optimal resource configurations and instructional design practices that promote effective e-leaming 
outcomes in K-12 learning environments currently are not recognized, generally understood, or agreed 
upon by e-leaming producers, consumers, and education policy leaders. Objective, researched-based 
guidelines and standards supporting the selection and screening of online courses are lacking. When 
seeking product information on online tools and advice about what works and what doesn’t in K-12 
e-leaming environments, school districts and state education agencies may be dependent upon the vested 
interests that are developing and selling online learning services. 

Recommendation 

❖ All concerned parties and agencies should support the development and diffusion of standards 
and assessment guidelines for online learning. Such standards and guidelines can assist local 
school districts and state education agencies with the selection and acquisition of well-designed 
and effective online learning. 

Finding 5 

Existing educational research and program evaluations that examine and analyze the outcomes and impact 
of online learning in K-12 learning environments presently are very limited. TTie few available research 
summaries and meta-analyses currently available do not include published data from recent program 
evaluations and assessments from state and federally supported virtual high school programs. 

Recommendations 

❖ Existing research summaries and meta-analyses concerned with e-leaming policy and practice 
should be expanded to include newly published findings on recent state and regional virtual 
high school projects. 

❖ Support for additional professionally designed and executed program evaluations and scientific 
educational research should be given a hi^ priority in all public and private agencies 
supporting effective implementation and use of online learning in K-12 learning communities. 

Conclusion 

In final analysis, e-leaming isn’t about digital technologies any more than classroom teaching is about 
chalkboards. E-leaming is about people and about using technology systems to support constmctive 
social interactions, including human learning. Although computers and other digital technologies clearly 
will play an increasing role in K-12 schools, e-leaming may work best when it is combined with some 
face-to-face classroom experience. In the best of all possible worlds, an eventual goal might be for 
students to have their own notebook computer to support both in-school and at-home learning, as long as 
they actively pursue publicly available educational opportunities. 

Books and traditional technologies certainly will continue to be important, along with telephones, 
satellites, compressed videos, audiotapes, and videotapes. But in all probability, today’s newest 
educational technology approaches — e-leaming and virtual schools — are destined to become tomorrow’s 
established instmctional delivery systems. New e-leaming technologies will become increasingly 
common for people in all walks of life, and increasingly integrated as an invisible and ubiquitous part of 
U.S. global, cultural, political, and economic systems. 
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The importance of a highly trained and skilled military has never been greater than today. Rising to meet 
this need is the capability to train personnel anywhere in the world at any time using distributed learning 
(TRADOC, 1999). In response, the purpose of this study is to understand how a blended or hybrid 
approach to ©-learning impacted the professional development of students in a high-level military course. 
This study addresses ©-learning from the perspectives of the course learners, the course advisor, and the 
instructors. Focus group discussions helped document distinct advantages and disadvantages from 
different components of the course. Issues and considerations for e-leaming mentioned consistently 
across groups should help with fiiture course design and delivery methods. In effect, this research might 
help in forming instructional design principles for the Web as well as the fine-tuning of this particular 
program and ones similar to it. 



Methodology 

At the U.S. Army Armor School in Fort Knox, Kentucky, the use of collaborative learning environments 
is taking center stage in the Armor Captains Career Course (AC3-DL) (Wardell & Paschetto, 2000). The 
purpose of the AC3-DL is to train captains to command companies and perform as assistant operations 
officers at command units such as a battalion. This training is conducted in three phases; the first two are 
online (Phase la: asynchronous, and then Phase Ib: synchronous), while the final phase is face-to-face. 

Procedure 

Interviews took place with two focus groups of four students who were completing the AC3-DL course as 
well as three instructors and the Distance Learning (DL) Education Advisor. Prior to the focus group 
meeting, the students completed a series of questionnaires related to their Internet backgrounds, attitudes 
toward working in groups, satisfaction and self-efficacy for the synchronous phase of the course, 
perceptions of the effectiveness of the online environment, and perceptions of task and interpersonal 
cohesiveness. 
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Results 



Across the students, instructors, and course supervisor focus group sessions, there were a number of 
distinct advantages and disadvantages noted. Naturally, the students liked the flexibility of completing 
assignments on at their leisure, the immediate feedback, and the thoughtful commenting and reflection of 
this environment. However, they mentioned problems with the length of most course modules, 
technology downtime or incompatibility, and system inflexibility. The instructors found that they liked 
the ability to tailor strategies to individual student needs, provide immediate feedback and online 
mentoring, standardize the content, embed small group interaction, update content, and foster knowledge 
application. However, they complained about the hi^ attrition, excessive time commitments for their 
students, and lack of instructor control over module size. The course designer noted that the online 
course can take track student learning, provide more authentic learning experiences, take advantage of 
advances in learning theory, and address individual student needs. However, she admitted that there were 
definite problems in these learning environments, such as the risks in conunitting to particular 
technologies or delivery mechanisms before the students begin to learn. 

Ten key Web-based instruction considerations or issues mentioned across participants are listed in Table 
1 . These considerations or issues relate to feedback, content meaningfulness, content size, course 
development and organization, the role of the online instructor, structuring small groups, flexible and 
active learning, technology utilization, assessment practices, and general skills such as online 
communication, problem solving, and teamwork. 



Table 1. Web-based Instruction Considerations and Issues 



Web-based 
instruction 
consideration or 
issue 


Student Advice 


Instructor Advice 


DL Education Advisor 
Advice 


1 . Feedback. 


E-mail is important 
mechanism for 
contacting instructors. 


Provide instant and 
consistent feedback with 
e-mail and other tools. 


Involve direct e-mail 
feedback. 


2. Meaningful 
and Real- 
World 
Content 


The construction of 
online products should 
approximate real-world 
application. 


Require students to 
produce products that 
instructors and peers can 
evaluate. 


Include meaningful 
content and allow 
students to apply new 
skills to real-life 
exercises. 
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Table 1 (Con’t) 



3. Size and 
Scope of 
Content 
Materials 


To maintain motivation 
and increase completion 
rates, divide 
asynchronous content 
and testing into smaller 
units or 

accomplishments. 


To increase student 
completion rates, 
instructors need some 
control to change the 
size of content modules. 


Utilize minimal 
extraneous content, 
graphics, or practice 
exercises. 


4. Course 

Development 

and 

Organization 


A pre-orientation session 
will help address 
questions and concerns 
about the online course. 
Students need lecture 
and direct instruction 
before project work. 


Learn basic content in 
asynchronous phase 
(“crawl”), put 
knowledge to use 
electronically and on 
paper in synchronous 
phase (“walk”), and 
apply knowledge in real- 
life scenarios in 
residential phase (“run”). 


Carefully analyze target 
audience wants and needs 
prior to course 
development. 


5. Role of 
Instructor 


Instructor is helpful as a 
facilitator of learning. 
The same instructor 
should support students 
across all phases of 
online training. 


Instructor role is more of 
a facilitator of the 
learning process; 
providing tools, means, 
and guidance to learn 
effectively. Indirect 
questioning, prompting, 
reminders, role playing, 
and direct requests are 
ways to engage and 
involve students. 


Instructor provides 
feedback and sense that 
someone cares about their 
learning. 


6. Small Group 
Structuring 


In online role-play, 
rotate roles among group 
members. 


Match stronger leaders 
and weaker students in 
role-play activities to 
boost performance and 
confidence. Provide 
instructions and 
information prior to 
online events such as 
role plays and product 
discussions. 


Create active 
environment with role- 
plays and simulations, but 
must provide balance 
between flexibility and 
learner accountability. 
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Table 1 (Con’t) 



7. Flexible and 
Active 
Learning 


Be flexible and allow 
students to complete 
online modules at their 
own pace; minimize 
need for instructor to 
certify students are 
ready for next step or 
phase. 


Distance learning helps 
Army Reserve students 
fit training into busy 
schedules and keep up 
with active duty 
personnel. 


Offer flexibility, choice, 
variety, meaningful 
contexts for learning, and 
student performance 
opportunities. 


8. Technology 
Utilization 


To minimize frustration 
and downtime, utilize 
basic functions or 
technologies, where 
possible. 


Use asynchronous 
communications for 
learning basic concepts 
and synchronous 
communications for 
application. 


Limit technological 
visions and begin to 
incorporate technology 
based on what it can 
presently accomplish. 


9. Build General 
Skills 
Through 
Online 

Communicati 
on, Problem- 
Solving, 
Teamwork, 
and Identity 


Small talk, 
introductions, and 
information sharing 
helps form team identity. 


Communication skills, 
problem solving, and 
teamwork are general 
skill outcomes of 
interactive distance 
learning. Online tasks 
should involve teaching 
students how to work 
with each other on a 
team to solve a problem. 
Teamwork and virtual 
talk among small groups 
fosters interaction and 
participation. 


Courseware structured to 
move from individual 
effort (asynchronous 
component) to 
application exercises in 
small group collaboration 
activities (synchronous 
component) to problem 
solving in collective 
efforts within units 
(resident component) is a 
useful framework for 
fostering student 
learning. 


10. Assessment 
Practices 


Online assessments 
should closely match 
real-world expectations. 
Focus might shift from 
quantity of learning or 
breadth across areas to 
quality or depth of 
learning in particular 
areas. Assessments 
should also cover 
smaller amounts of 
instruction or learning. 


Asynchronous learning 
is more ideally suited for 
objective tests and 
measurements, while 
synchronous might be 
used for student 
performances or 
products and criterion 
referenced evaluations. 


Online assessments can 
include automated pre- 
tests, post-tests, and 
practice exercises that 
provide immediate 
student feedback. 
Random feedback or 
assessment tools are also 
beneficial. While 
evaluation gates require 
application of learning, 
the learning management 
system needs to be more 
flexible and adaptable in 
regards to items missed 
on gate examinations. 



While this focus group study provided some general information about the keys to success and failure on 
the Web, the coding of over 6,500 chat acts indicated that there were shifting patterns of interaction 
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during the synchronous phase of the training. For example, while technology concerns gradually 
diminished, on task discussion peaked in the middle months and social interactions were higher at the 
start and end of the training. Overall, student chats were categorized as on-task 55%, social 30%, or 
technology-related 1 5%. 



Final Comments 

There are many avenues for course and tool development as well as student testing and evaluation within 
military e-leaming as well as in higher education, K-12, and corporate settings. The present study 
provided one look at the advantages and disadvantages as well as many instructional considerations and 
issues within a unique online learning program. Other studies might explore completion rates, attitudes, 
and overall learning when one’s career is not contingent on course completion. The years ahead will 
require a myriad of contributors, many focus group discussions, thoughtful critique of what works and 
what tends not to work, extensive teaching and research experimentation, instructional fortitude, and 
careful project planning and funding. 
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Faculty Preparation: 

Getting Started in eLearning 

Lynnda L. Brown 
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C. Beth Henderson 
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Tulsa Technology Center 

Internet-based distance learning has become popular among educational institutions as a method of 
offering courses and degree programs in place of campus-based face-to-face instruction. As distance 
learning has proliferated at a dizzy pace in all segments of education and training, the need for instructor 
training in teaching at a distance has become obvious. Student consumers have begun to discriminate 
among those courses that have simply been transported to the online environment without modification 
and those courses that have been modified or specifically designed for teaching at a distance. 

As more and more instructors are being asked to place their courses online, they are concerned about how 
to effectively design their courses so that the highest level of student achievement and satisfaction is 
attained. Additionally, many faculty have found themselves with little or no experience with online 
teaching/leaming and are often uncomfortable and intimidated in this new environment. 

The purpose of the Getting Started in eLearning Workshop was to provide a vehicle for educators 
beginning their online teaching experience to become comfortable in this environment and to develop 
strategies for success in their first online courses. The workshop focused on helping faculty understand 
the need for modification of current course design for delivery in the online environment and provide 
strategies for the effective modification and delivery of their content. Three areas of instructional design 
were addressed. They were building an online learning community, content delivery, and interaction. 

Professional Development Strategies 

The Getting Started in eLearning Workshop incorporated two professional development strategies: - 
(a) the mastery of specific concepts instead of exposure to many concepts, and (b) the importance of 
building background and experience in a new learning environment. It followed a hybrid course model 
(combining an online component and a face-to-face component). 

Professional development specialists often face a dilemma in the instructional design of workshops. 
Faculty understand the importance of the lesson cycle in their own classroom (concept introduction, 
guided practice, independent practice, and conclusion), but often want the “down-and-dirty” when it 
comes to their own learning. All too often the tendency is to cram as many skills as possible into the time 
allowed, with no time for practice or application. 

Our experience has shown that in the area of technology-based professional development, the courses that 
focused on the mastery of a few skills were more effective than courses that simply provided exposure to 
many skills. Practice and application of technical skills gave faculty confidence in their ability to use the 
technology effectively in their classroom. Practice and application is especially important when dealing 
with the online environment. Providing technical skills without a pedagogical use for the technology can 
be devastating in an online classroom. 
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The Getting Started in eLearning Workshop was designed for faculty who were just getting started 
teaching in the online environment. Many of them had never seen an online class and had trouble 
conceptualizing anything other than posting their PowerPoint presentations and lecture notes for students 
to download and to read. The online component of the workshop provided a chance for these faculty to 
experience an online class as a student. This was essential in building background for the potential of the 
online classroom. Not only did the faculty experience different strategies for content delivery and 
interaction, they also experienced the frustration of not having the technology work the way it should and 
recognized the amount of communication needed to encourage a frustrated student. 

A hybrid professional development offering has many advantages. Readings and background-building 
material can be provided in the online environment. Thoughtful discussion can take place in a discussion 
area. . .all on the faculty’s “own” time. Once background has been built and reflected upon, participants 
can come together in a face-to-face session where the technical and more in-depth applications of the 
concepts can be addressed. 

Overview of Getting Started in eLearing Workshop 

The idea for the Getting Started in eLearning Workshop developed through a review of current literature 
in the area of effective instructional design for online courses. Three critical areas formed a common 
thread throughout the literature. They were the importance of building an online learning community, the 
necessity in providing multiple modes of content delivery, and the need to plan specific and required 
opportunities for interaction within the scope of the course. Also, apparent in the literature was the lack 
of faculty awareness of these three critical areas and/or effective strategies with which to incorporate 
them. 

In addition, faculty had been asking for a workshop on how to use online courseware (Blackboard). They 
wanted to know how to get their content online, change colors, make links, use the gradebook, etc. While 
technical knowledge and skill was a recognizable need, a pedagogical basis for the use of various 
technological components available on Blackboard was also needed. 

As dialog and planning progressed for the workshop, it became clear that a traditional workshop would 
not effectively meet the goals set. Faculty needed to develop background experience in an online 
environment before they could understand the instructional design strategies to be discussed. An online 
demonstration component of the class seemed the ideal solution. After sharing several common 
experiences with community building, content delivery, and interaction from the perspective of a student, 
faculty would better understand the pedagogical theory behind the instructional design strategies that 
would be discussed in the face-to-face session. They would also have experience with the courseware 
interface and its technological possibilities, providing background for the technical “how-to.” 

Online Demonstration Component 

The purpose of the online demonstration component of the workshop was for participants to experience 
various instructional and technological delivery methods for Internet-based distance educations courses 
from a student’s perspective. Through their experiences as students, a better understanding of effective 
instructional and technological delivery methods were discovered and internalized. 

The online component consisted of four weeks of activities. Activities were posted on Sunday. 
Participants completed the activities during the week. The activities were designed to take approximately 
one hour to complete. Each week the activities modeled different strategies in community building, 
content delivery, and interaction. Sometimes the activities covered more than one area. 
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Week One Activities included a student orientation of the Blackboard course site, building a student 
homepage, and participating in a discussion board. The orientation addressed both community building 
and content delivery. Students became familiar with their classroom and discovered if they could 
navigate external links, download a document, and email their instructor. The student homepage also 
addressed community building as it provided an opportunity for students to become familiar with their 
classmates. The discussion board topic (which addressed community building as well as interaction) 
asked students to share why they signed up for the Getting Started in eLearning Workshop and what they 
expected to learn in the class. Week One Activities addressed interaction in all three of the critical areas: 

»t» student-to-content; student orientation 
»> student-to-teacher; email, discussion board 
<» student-to-student; student pages 

Week Two Activities included a look at PowerPoint presentations that covered the same content in two 
different presentation design strategies. Browsing external web sites, an online quiz, and a discussion 
board posting summarizing an online article of choice were also included. Community building was 
addressed through the content of the PowerPoint presentations and external web sites identifying qualities 
that make a successful online student. Content delivery strategies included an examination of PowerPoint 
design, utilization of external links, and an online quiz. Interaction focused on student-to-content and 
student-to-teacher as participants chose a topic, read an article, and summarized it on the discussion 
board. 

Week Three Activities focused on ways interaction and content delivery could be combined. Participants 
were asked to read two of the article summaries posted on the discussion board and post a response that 
included questions for further information. A session in the virtual classroom (chat) was also held. The 
virtual class session was offered at two different times during the week providing options for the 
participants. Rules of engagement to be used during the session were posted before hand and followed 
during the session. The session topic centered around the online experience to date and the instructional 
design concepts experienced. 

Week Four Activities focused on various strategies for content delivery. Instructional design content was 
presented as a video, as a text document, and as a web page where the information had been “chunked” 
with Flash learning objects included. Interaction on the discussion board focused on student-to-student 
interactions as participants responded to the questions posted during Week Three and student-to-teacher 
interactions as they posted their reactions to the content delivery strategies. 

Face-to-Face Component 

The purpose of the face-to-face component was to examine the theory behind instructional design 
strategies of online courses (or an online component to a face-to-face class) in order to determine which 
designs should be included in future course development. This component also provided technical 
“know-how” needed to set-up a course using the Blackboard courseware interface. 

The six-hour face-to-face workshop was divided into two distinct segments. The first dealt with 
instructional design principles. Participants divided themselves into three teams to explore one of the 
three design principles that were focused upon during the online demonstration class (learning 
community, content delivery, and interaction). The teams then split up and explored pre-selected 
resources to gain practical information on their chosen topic, focusing on things to do or not to do. (The 
resource list can be found at http://www.eighthfloor.org/resources/websites/online.htm.) After exploring 
the resources, teams gathered back together to synthesize information and to prepare a group presentation. 
Teams reported back to the whole group, and an analysis occurred of how that principle of instructional 
design was applied in the online component. 
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The second segment dealt with the technical creation of a Blackboard course site. Blackboard.com was 
utilized as faculty from multiple institutions were involved in the training. Participants individually 
created a course using components pre-developed for the workshop. A tutorial was prepared which 
guided participants through steps of course development including: creating of the course, modification 
and development of course structure, addition of instructional content (including basic HTML, web- 
authoring software, multimedia components, and zipped files), and enrollment/modification of students. 

Workshop Results 

Overall, the hybrid workshop has been very effective. Participants truly appreciated the background 
building and student perspective that they gained during the online component of the workshop while the 
face-to-face component provided motivation for an exploration of resources and an opportunity for 
“personal” interactions. 

Comments from the online demonstration component of the workshop showed many of the activities 
modeled to create a learning community were helpfiil, especially the use of the student homepages and 
the technical support discussion board. An outstanding discussion resulted from the different strategies 
involved in content delivery. Participants were able to identify a strategy that worked best for them, but 
were also able to see that multiple strategies would be most effective in a class. Participants, however, 
were most impressed with the different interaction strategies presented. They liked the structure of the 
discussion boards, which focused on course content and did not get “off target.” The rules of engagement 
for the virtual classroom helped many see that a synchronous discussion at a distance could be 
manageable and beneficial. 

During the face-to-face component of the workshop, participants enjoyed the format of the instructional 
design activity. They liked exploring web sites on their own and discovering what the “experts” had to 
say about instructional design. Working in teams was an effective motivator. Animated discussions took 
place during the group presentation planning and during the presentations themselves. Participants were 
pleased with the technical tutorial with pre-made components, which allowed them to focus on the 
technical aspect of Blackboard without worrying about content or design. 

The overall feeling of the group could be summed up by one participant’s comment: 

This workshop did an excellent job of exposing us to a wide variety of issues in online 
instruction. I think whatever I manage to get together for the course I’m teaching . . . will 
be greatly improved as a result of this workshop. I can also clearly see the need for 
ongoing tinkering in an online course and the CRUCIAL need for availability of the 
instructor and /or technical support people. 
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The Wisconsin Online Resource Center Digital Library 

In 1999, the Wisconsin Online Resource Craiter (Wise-Online Digital Library) is the foundation from two 
iimovative projects that have successfully developed and demonstrated a unique approach to impact the 
tftaf-hin g and learning effectiveness of web-based curriculum and course design in post-secondary 
education. Wise-Online was funded as an initial project of the FIPSE Learning Anytime Anyplace 
Partnership to design and develop a digital library of learning activities (learning objects) that correlate 
with the common competencies of the nine core courses of the WTCS General location curriculum. 

Faculty content experts from across the state are trained to formulate creative approaches to teaching and 
learning of their course competencies that optimized the power of the new multimedia and web-based 
technology into dynamic and interactive online learning activities. The ideas were shared with a 
multimedia instructional designer who then worked together with a web technician to technically design 
into a small unit of learning known as a learning object. The completed learning objects are placed in the 
digital library www.wisconline.org. Metadata tagging (designed to be SCORM, IEEE compliant) is used 
added to learning object to ensure that it can be searched in the following ways within the Wisconsin 
Online Resource Center Digital Library: by key word, by content area, by topic area, by author, by 
learning style. The learning objects can be linked to course competencies that can be used and reused as 
an enhancement or alternative to traditional teaching of competencies, including alternative learning 
styles and assessment options. The learning objects can also be designed and use as an aggregation of 
multiple objects into larger collections and reus^ in a variety of applications. 

Faculty enthusiasm for this collaborative approach to developing learning activities and integrating 
technology into the classroom spread to the WTCS manufacturing faculty who envisioned a digital library 
to support the core concepts and courses that support post secondary and adult basic education throughout 
the sixteen technical college districts in Wisconsin. In 2000, the National Science Foundation - 
Advanced Technological Education Program funds a three year WTCS project to develop the Core 
Manufacturing Digital Library component of the Wisconsin Online Resource Center. Highly technical 
content and laboratory procedures correlated with core competencies are able to be captured using 
creative online features that integrate a wide variety of interoperable multimedia options including video, 
graphics, animation and interactive assessment and instructor communication techniques. Those faculty, 
who have not been the most comfortable with technology, have embraced this approach as an effective 
way to teach the most complicated and confusing topics as well as being a “safe” introduction to 
instructional technology that emphasizes frieir content, not computer, expertise. 

The student reaction has been overwiielmingly positive and impatient to have this learning approach 
available throughout their courses in every academic area. The seamless marriage of instruction and 
technology has been very effective in helping students learn the most difficuh concepts and appeals to the 
increasingly computo* literate student population. 
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Web-based resources are designed, built and linked to competency-driven/topic related curriculum. The 
digital library is developed with the collaboration of content-expert faculty and instructional technology 
specialists. The implementation of this collaboration is facilitated by faculty development to design 
multimedia based chunks of learning (learning objects) that can stimulate and accelerate student 
understanding of core concepts. 

The project team for Wise-Online Digital Library has witnessed how faculty who have become involved 
in the development of learning objects see this new concept as a viable part of the curriculum 
development and design processes. With instructional technician support, faculty are able to have their 
content moved from an analog version to web-based, high quality, peer-evaluated learning resources 
created for easy access and use. The learning objects that have ^en created by faculty who are tackling 
those “hard to teach” concepts are finding that a visual, multi-media approach is helping to increase not 
only learners’ understanding, but is accelerating the learning in many cases. 

Faculty have appreciated the fact that the project dollars support instructional technicians who are able to 
take the content provided by the faculty authors and web the content into multi-media learning objects 
utilizing the Macromedia Flash software. The learning objects are then housed in a searchable digital 
library for easy access and may be used by other faculty from any of the sixteen colleges. As of May 
2002, over 1 00 faculty representing 1 5 technical colleges have authored over 500 manufacturing learning 
objects with another 200 in development. 

The Wisconsin Online Resource Center Digital Library is found at www.wisconline.org. 
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The Wisconsin Online Resource Center defines a learning object with the following characteristics: 

❖ A new way of thinking about learning content — Traditionally, content comes in a several-hour 
chunk called a course. Learning objects are much smaller units of learning, ranging, for 
example from 2 to 15 minutes. 

❖ Small, independent chunks of knowledge or interactions stored in a database — can be presented 
as units of instruction or information. 

❖ Based on a clear instructional strategy — intended to cause learning through internal processing 
and/or action. 

❖ Self-contained — each learning object can be taken independently. 

❖ Interactive — each learning object requires that students view, listen, respond or interact with the 
content in some way. 

❖ Reusable — a single learning object may be used in multiple contexts for multiple purposes. 

❖ Able to be aggregated — learning objects can be grouped into larger collections of content, 
including traditional course structures. 

❖ Tagged with metadata — every learning object has descriptive information allowing it to be 
easily found by a search. 

♦♦♦ Built to meet the Wisconsin Online Resource Center Quality Standards. 

♦I* Learning objects let you have learning that is: 

❖ Just enough — if you need only part of a course, you can use the learning objects you need. 

❖ Just in time — learning objects are searchable, you can instantly find and take the content 
you need. 

❖ Just for you — learning objects allow for easy customization of courses for a whole 
organization or even for each individual. 

Digital barriers currently exist for a number of faculty and students within our country. With the demand 
and need on our educational institutions to integrate technology and adapt to technology, while at the 
same time institutions are faced with rising costs, the balancing act is challenged. The Wisconsin Online 
Resource Center is an open and free digital library for faculty to access and use. What become available 
to them are high quality, web-based resources through the Internet. Many faculty, within the Wisconsin 
Technical College System, who are teaching in store front operations or in incarcerated locations face the 
same barriers with the lack of hardware and access to the Internet. The Wisconsin Online Resource 
Center staff began to create CD ROMs with learning objects in order that faculty may use the learning 
objects with their students. 

Multiple barriers continue to limit the integration of technology into the post-secondary curriculum. For 
most faculty members, it is neither appealing to adapt an already developed online course “off-the-shelf’ 
that is created by another instructor or company. Most instructors are more comfortable with customizing 
their teaching and learning experiences for their students. In addition, there are often restrictions to using 
only portions of already developed courses or modifying certain components to meet specific needs. 
Practically speaking, existing online or digital resources often are prohibitive due to cost, uninviting 
password requirements or are instructionally incomplete having little meaningful use without major 
technology intervention, which effectively widens Ae digital divide. This factor inhibits the most 
disadvantaged individuals and institutions from access to dynamic and powerful technology-based 
resources to enhance those academic strategies where they are most needed. 

Most educators find many of the digital collections or libraries are actually “flat files” of images or video- 
clips that in most cases have not been enveloped by comprehensive teaching and assessment aspects that 
lend them to be readily used and reused by faculty in their content area. The role of the instructional 
designers and technicians in higjier education has been compartmentalized and often relegated to being a 
webmaster who is not connected with curriculum or student learning. Many assist with the development 
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of online courses but are limited to using a singular software package to house the digital content. Rarely 
are they free to collaborate directly with faculty and access the plethora of digital solutions that can best 
convey a creative instructional idea in the most powerful pedagogical and technological manner. 

The fundamental principle of the Wisconsin Online Resource Center approach is that faculty members 
value online educational materials that are peer-evaluated and collaboratively designed but not an entire 
course. The “buy-in” to this approach to instructional technology integration demonstrated by faculty 
members has created a growing demand in Wisconsin and beyond for training to expand the development 
of learning objects to new academic areas. Copyright and intellectual property rights that frequently 
emerge, as issues in the online educational arena have become a moot point due to the overwhelming 
faculty developers who have authored learning objects for Wisconsin Online Resource Center have belief 
in value of shared resources. The project team members respect the faculties’ interests and desire to share 
their ideas, and are not interested in profiting from the creation of learning objects. With this philosophy, 
the copyright is held by the authoring faculty member’s home college with a nonexclusive license to share 
and disseminate the learning object through the Wisconsin Online Resource Center. Educators are 
encouraged to use, reuse, evaluate and give feedback, but protection is assured from copyright 
infringement for unintended purposes. 
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Analyze Motivation-Hygiene Factors to Improve Satisfaction 
Levels of Your Online Training Program 

Seung Youn (Yonnie) Chyung, Ed.D 
Visiting Assistant Professor 
Instructional & Performance Technology 
College of Engineering 
Boise State University 

Introduction 

One of the problems in Internet-delivered online instructional programs is high attrition. Despite the 
various potential benefits of online instruction, online learners tend to drop out of online instruction much 
earlier or more frequently than they do in traditional classroom instruction (Kember, 1995). There are 
various reasons for dropouts. For example, learners may stop attending on line classes because they think 
they have learned enough. Learners may stop enrolling in online courses because they have financial 
difficulties to support their continuous education. Learners may quit learning in online instruction because 
they are not satisfled with their online learning experience. It is critical to analyze the root causes of 
attrition before designing effective interventions to combat attrition (Chyung, 1999, 2001a, 2001b). If 
learners express dissatisfaction toward the online programs, the factors that cause dissatisfaction should 
be further investigated and eliminated. While isolating factors that cause dissatisfaction toward online 
instruction, it is also important to recognize factors that satisfy online learners and to continue to utilize 
those motivating strategies in online instruction. A helpful theory that guides such root cause analysis 
processes is Herzberg’s motivation-hygiene theory. 

This article is based on a 3-year long online training evaluation case study. In this article, Herzberg’s 
motivation-hygiene theory will be introduced as the theoretical framework. Results obtained from the 
three-year case study will be presented. Based the results, hygiene factors that cause dissatisfaction 
toward the online training program and motivation factors that cause satisfaction toward the online 
training program will be explained. At last, general guidelines for conducting ongoing evaluations and 
continuously improving the motivational appeal of online training programs will be discussed. 

Herzberg’s Motivation-Hygiene Theory 

In the 1950s, Frederick Herzberg and his colleagues studied factors that affected job attitudes. In one of 
their studies with engineers and accountants in the Pittsburg, Pennsylvania area, they interviewed 
individual employees and asked them to describe “sequences of events” happened on the job where they 
felt exceptionally good or bad about their job. Analyzing the interview data, “thought units” were 
identified. A thought unit was defined as “a statement about a single event or condition that lead to a 
feeling, a single characterization of a feeling, or a description of a single effect” (Herzberg, Mausner & 
Snyderman, 1959, p. 38). Thought units were then categorized according to similarity. The frequency 
percentage of positively or negatively stated items found in each category of thought units was used to 
determine the degree and direction of employees’ attitudinal status toward their job. From their research, 
they concluded that there were two different sets of factors that affected positive or negative job attitudes. 
A set of factors such as achievement, recognition, work itself, responsibility, and advancement affected 
positive job attitudes. They called such factors motivation factors. Another set of factors such as working 
conditions, company policies, relations with supervisors, subordinates or coworkers, and pay affected 
negative job attitudes — ^they called such factors hygiene factors. The motivation-hygiene theory was 
established based on the research findings (Herzberg, 1 966). 
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The motivation-hygiene theory has given a new perspective on job attitudes. The traditional perspective 
on job attitudes was that the opposite of job satisfaction was job dissatisfaction and the opposite of job 
dissatisfaction was job satisfaction. In other words, removing the causes that make workers feel 
dissatisfied would help them feel satisfied and removing the causes that help workers feel satisfied would 
make them feel dissatisfied. Herzberg’s motivation-hygiene theory suggests that it is not true. In his 
theory, Herzberg explains that the opposite of satisfaction is not dissatisfaction but no satisfaction (i.e., 
neutral to either satisfaction or dissatisfaction), and the opposite of dissatisfaction is not satisfaction, but 
no dissatisfaction (i.e., neutral to either satisfaction or dissatisfaction). When the condition of hygiene 
factors deteriorates to a level where employees perceive it to be unacceptable, it causes job 
dissatisfaction. But improving the condition of hygiene factors to an optimal level does not necessarily 
make employees feel satisfied with their job. It is primarily the motivation factors that improve 
employees’ job satisfaction (Herzberg, 1966, 1987). 

Applying Motivation-Hygiene Theory to Online Training Programs 

The conceptual model presented in Herzberg’s motivation-hygiene theory that distinguishes hygienic and 
motivational factors that affect job attitudes is applicable to analyzing learners’ attitudes toward online 
training programs. 

In order to help online learners develop positive attitudes toward online training, one of the first things to 
do is to analyze what causes them to feel negatively about the online training program and to try to 
improve the condition of those hygiene factors. While maintaining the condition of the hygiene factors on 
an acceptable level, online instructors should recognize factors that tend to help online learners feel 
satisfied with the online training program and continue to utilize those motivating instructional strategies. 
When maintenance strategies and motivational strategies are systematically implemented, online learners’ 
satisfaction levels will improve. 

Described in the following section is a 3-year long case study conducted at Boise State University's 
Instructional & Performance Technology department. Similar methodologies used in Herzberg’s research 
were adopted in the case study in order to reveal motivation factors that affect learners’ positive attitudes 
toward online training and hygiene factors that affect negative attitudes toward online training. 

Case Study: Analyzing Attitudes Toward Online Instruction 

The Instructional & Performance Technology (IPT) department of the College of Engineering at Boise 
State University offers its entire master’s degree program on line. The IPT online courses are delivered 
via Lotus Notes. The IPT department provides a week-long asynchronous online technical training 
program (a.k.a., IPT Boot Camp) to new online participants prior to their first semester. The IPT Boot 
Camp program is designed to teach them basic technical knowledge and skills for using Lotus Notes in 
IPT online course databases, and help them feel confident and become successful online learners (see 
Figure 1). 

The IPT Boot Camp program is delivered a week prior to the first day of each of Spring, Summer and Fall 
semesters. Between Spring semester, 2000 and Summer session, 2002, eight (8) IPT Boot Camp 
programs were delivered to new online participants. At the end of each IPT Boot Camp program, the IPT 
Boot Camp evaluation was administered via the web. The evaluation questionnaire consisted of the 
following forced-choice questions (using a 5-point Likert scale) and open-ended questions. 
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SD D N A SA 



• During the IPT Boot Camp, I learned the basic skills to utilize Lotus Notes database 
features. 

► The IPT Boot Camp task directions were clearly stated. 

► The number of tasks provided in the IPT Boot Camp was appropriate. 

► I feel prepared to use Lotus Notes database as a learning tool for my first IPT online 



SD D N A SA 
SD D N A SA 
SD D N A SA 



SD D N A SA 



course. 

• The IPT Boot Camp experience was valuable to me. 

► While you were completing the IPT Boot Camp tasks, was there a fhistrating moment? If so, please 
describe. 

» What components of the IPT Boot Camp program are you satisfied with? Please describe. 

» Please provide any other comments you have about the current IPT Boot Camp program, or suggestions 
you may have for future IPT Boot Camp sessions. 
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Table 1 . Demographic Information of the IPT Boot Camp Evaluations 



Year 


2000 


2001 


21 


m 


Total 


Semester 


Spring 


Summer 


Fall 


Spring 


Summer 


FaU 


Spring 


Summer 


Number of IPT 
Boot Camp 
Evaluation 
Respondents 


14 


9 


29 


44 


17 


32 


46 


19 


210 


Number of IPT 
Boot Camp 
Participants 


53 


16 


38 


48 


17 


32 


46 


19 


269 


IPT Boot Camp 
Evaluation 
Return Rate 


26.4% 


56.3% 


76.3% 


91.7% 


100% 


100% 


100% 


100% 


78.1% 



Two hundred ten (210) out of 269 IPT Boot Camp participants during the eight (8) IPT Boot Camp 
programs from Spring 2000 and Summer 2002 responded to the IPT Boot Camp Evaluation questionnaire 
(see Table 1). By analyzing responses to the open-ended questions, ‘thought units’ were identified. The 
thought units were then sorted into different categories according to their similarity. Events that affected 
learners to feel negatively during the IPT Boot Camp program were identified as hygiene factors. Events 
that helped them feel satisfied with the IPT Boot Camp program were identified as motivation factors (see 
Table 2). 



Table 2. Motivation Factors and Hygiene Factors Found in IPT Boot Camp (Based on Responses to 
Open-ended Questions) 



Motivation 

Factors 


« Learning itself (i.e., learning new software) 

• Feelings of confidence (e.g., feeling prepared to start a new online program; 
getting more familiar with the environment) 

• Effectiveness of instructional materials (e.g., clearly written, well structured, 
easy to follow, multimedia) 

• Social Interactions (e.g., opportunities to interact with other participants) 

• Instructional feedback provided by the instructor (e.g., the information, the 
speed) 

• Flexibility/convenience provided in the learning environment (e.g., alternative 
Web access, asynchronous method) 


Hygiene 
. Factors 


• Required amount of time to complete tasks (e.g., long downloading time) 

• Technology-related events (e.g., unexpected technical problems, a computer 
crash, a drink spilled on computer, a malfunctioning printer, web browser 
incompatibility) 

• Required amount of cognitive effort to complete tasks (e.g., things that made 
learning more difficult than expected, usually due to the unfamiliarity to or the 
complexity of the online environment) 

Learner preferences on instructional design (e.g., the instructional sequence or 
the components of media used in instruction) 



A Summary 

Herzberg’s motivation-hygiene theory suggests that motivation factors and hygiene factors are two 
different sets of factors that influence job attitudes. Herzberg explains that motivation factors are related 
to what people do on the job while hygiene factors are related to the context or environment in which they 
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do their job. Presence of hygiene factors in a poor condition causes Job dissatisfaction and maintaining 
hygiene factors on an optional level does not necessarily increase Job satisfaction levels. Presence of 
motivation factors influences Job satisfaction levels. 

The conceptual model presented in Herzberg’s motivation-hygiene theory was applied to analyzing 
factors that affect learners’ attitudes toward the IPT Boot Camp program. The 3 -year evaluation results 
seem to support Herzberg’s research findings that there are two distinct sets of factors, one of which 
affects learner satisfaction and the other one of which affects learner dissatisfaction. For example, a long 
downloading time was identified as a hygiene factor. However, there was no evidence that absence of the 
poor hygiene condition would increase learners’ motivation levels. The evaluation results also support 
Herzberg’s idea that hygiene factors are maintenance factors that are environmental (e.g., downloading 
time, technical problems, etc.) whereas motivation factors are contributors to psychological growth (e.g., 
confidence, accomplishments, learning itself, etc.). 

Recommendations: Rapid Processes of Evaluation and Intervention Design 

It is critical to conduct ongoing evaluations on the effectiveness of online training programs for 
continuous improvement. An evaluation conducted at the end of a program (a.k.a., a rear-end analysis) 
will reveal learners’ perception toward the training program, whether positive or negative. The rear-end 
analysis data will be fed back to the front-end analysis phase of the following cyclical revision processes 
and help redesign and improve the quality of the online training program. In order to improve learners’ 
attitudes toward an online training program, it is recommended to implement cyclical rapid processes of 
conducting an evaluation, analyzing the motivation and hygiene factors, and revising the program 
accordingly. 

Herzberg’s motivation-hygiene concept is an effective guideline when analyzing causes of online learner 
retention and attrition. To be systematic, it is important to put initial effort on improving the condition of 
hygiene factors before implementing motivational strategies. While maintaining the optimal level of 
hygiene factors, additional effort should be put in order to improve the motivational appeal of the online 
instruction. A helpful guideline to enhance the motivational appeal of online instruction is John Keller’s 
ARCS model. Online instruction can be redesigned by addressing the four factors; Attention, relevance, 
confidence, and satisfaction (for more information about the strategies, see Chyung, 2001b). 
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Finding the Right Blend of Learning at PNC Bank 

Alan L. Coates, Training Design Specialist 
Regional Community Bank Training Design 
PNC Bank 

Christopher J. Taylor, Lead Instructional Designer 
Regional Community Bank Training Design 
PNC Bank 

Introduction 

As companies expand in size and territory, training organizations are faced with the challenge of ensuring 
workforce competency while controlling the costs associated with the development of those 
competencies. PNC Bank is no different. We needed to control turnover, reduce expenses, and develop 
a workforce of 10,000 stretched across 6 states. We had to ensure that all employees had the skills, 
knowledge, and attitudes to be successful. PNC Bank’s Regional Community Bank (RCB) Training 
department needed to find the right blend of training for our employees in order to satisfy them, while 
conducting training as economically as possible. The example that we will share with you is our Teller 
New Hire Program. 



The Challenge 

We were faced with three major challenges in developing the Teller New Hire curriculum. First, how 
would we build a foundation for success using a competency-based front-end analysis? Second, how 
would we determine the best method of delivery for each competency, as well as the best format? Third, 
how would we coordinate multiple project teams so that instructional designers would focus on their tasks 
and not their turf. 

Step i: Build a foundation for success through an extensive, competency based, front-end analysis 
using five key factors 

❖ Define business need: In today's knowledge economy, organizations that understand and make best 
use of their knowledge capital are in the best position to thrive. Assessing what you know — and 
what you need to know — is a critical and crucial step in organizational growth as well as determining 
business need. 

♦J* Organizational culture readiness: A major benefit for an organization is the ability to protect core 
competencies by finding qualified candidates to replace highly skilled workers and senior managers 
due to attrition. 

Align skills with objectives and goals: Create job profiles and map to a specific set of skills and 
proficiencies, recommend specific learning to bring skills in line with your business goals, establish 
minimum proficiency requirements for a job profile, and update a person's skill proficiency upon 
completion of course or class. 

❖ Improve strategic planning and overall strength: identify the skills that your organization needs to 
stay competitive, frilly leverage the knowledge, skills, and competency of your people, and make 
sure you have the right mbc of skills to meet your business goals. 

❖ Individual career development: Allows the person to determine their skill gap, enables manager, 
subordinates, and peers to assess their skills (360 degree assessment), creates a plan to manage skill 
development, and identify job profiles and the skills associated with the requirements. 
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Step 2: Develop the RCB Teller Competency Model 

The first step was to identify the teller competencies and list the key behaviors and tasks associated with 
those competencies. See table 1 for a list of the identified competencies and key behaviors. 

Table 1 



Teller Competencies 



Key Behaviors and Tasks 



• Focus on clients 

• Work efficiently 

• Solve problems 

• Manage time 

• Build relationships 

• Apply compliance knowledge 

• Maintain technical knowledge 



• Interpersonal skills 

• Listen and speak effectively 

• Demonstrate Service Guarantees 

• Be aware of branch activity 

• Recognize client servicing and product needs 



The teller competencies would be located within eNN, our learning management system. Based on the 
competencies, we developed a curriculum of the required courses to give the new teller the needed skills 
for success. Next, the delivery method needed to be identified. 

Step 3: Delivery Methods 

Examining the advantages and disadvantages for each delivery method, we structured the curriculum to 
be a blend of classroom and technology-based instructional delivery. Table 2 outlines the thought process 
that went into designing the structure: 

Table 2 



Course Structure 



• Technology-Based 

-> Introduction that relates training to business 
goals. 

-> Pre-test 

-> Job-specific modules 
-> Post-test to measure proficiency 
-> Remediation 



• Classroom 

-> Modules that are task oriented 
— > Instructors introduce lesson and objectives 
— > Employees participate in exercises and role 
playing 

— > Embedded questions for self-checks. 



The course map. Table 3, outlines how each new hire receives 7 days of training using technology. 
Courses such as Basic Banking define terms associated with the financial services industry. Consumer 
Products gives the new hire the information pertaining to the products that they service and sell. 
Compliance reviews the regulations that the new hire must adhere to in their daily business transactions. 
The systems course allows the new hire to learn the system functions that they will need to use on a daily 
basis. All of these courses are completed on-line, with classroom follow-up and hands-on assessments. 
See Table 3 for a list of the curriculum. 
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Table 3 



Teller Course Map 

Day 1 RCB Orientation 

• Introduction to eNN 

Day 2-7 In-Branch Experience 

• Basic Banking CBT 

• RCB as a Business CBT 

• Consumer Products CBT 

• Compliance CBT 

• Negotiability and Endorsements CBT 

• A Day in the Life of a Teller CBT 

• Service Guarantees CBT 

• Systems CBT 

• Debits and Credits CBT 

• Security and Cash CBT 

• Maintaining a Cash Drawer CBT 
Day 8-12 Classroom 

• Hands-on job specific simulation and role play in a classroom setting. 

Step 3: Project Team Coordination 

Set the scope of the project with the different partners. In this case, PNC RCB training and Pathlore were 
the partners. PNC provided project management and subject matter experts, which included the 
information technology piece of the project. Pathlore provided project management, designers and 
authors and technical experts. Using a team approach, we worked together to develop this blended 
solution curriculum. Each team member was accountable to the core team to develop his or her 
respective piece(s) of the curriculum. Through a free exchange of ideas and updates, we were able to 
bring all aspects of the development together, on time. 

Conclusion 

From executive sponsors to the end-users, each played a vital role in developing a program that towered 
the cost to deliver this training, while increasing the retention of the lessons. We identified the overall 
business needs, objectives and strategies. The instructional design tasks included the development of the 
instructional goals, the delivery methodologies, and the instruction sequencing. Additionally, we 
developed lesson plans, the material to support the plans and invested in the technology required to run 
the CBT courses. All of these together were needed to ensure success. Now, how can we improve on this 
curriculum? 
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Management and Delivery of Technical BS Degree Completion 
and MS Degree Programs to Distance Learners 

Dr. Gerald W. Cockrell 

Professor and Director of the MSECT Program 

Dept, of Electronics and Computer Technology 
Indiana State University 

Ms. Nancy A. Pearson 
Coordinator of Distance Learning 
Division of Lifelong Learning 
Indiana State University 

This session will include a presentation on the management and delivery of a distance delivery technical 
bachelor’s degree completion program and master’s degree program. The management portion of this 
session will cover topics such as admissions, student advisement, and coordination of distance delivery. 
The course delivery section will consider the unique aspects of delivering a technical, laboratory-based 
program to distance learners. Methods used to transfer knowledge and experiences in a virtual laboratory 
environment using simulation and lab-in-a-box techniques in terms of student services will be considered. 

This session will center on the unique relationship between a remote campus representative and the 
distance program director and course professor. Indiana State University has a number of programs 
delivered using distance education methods at the bachelors and masters level. Specific programs are 
technical in nature, including a bachelors degree completion program and a master degree, both in 
electronics and computer technology. Both programs provide some unique challenges such as the delivery 
of technical and laboratory-based course content. The increased number of students interested in these 
types of programs has created a void in the ability to service these students with advising, registration, 
and laboratory coordination activities. To provide these vital student services to students enrolled in the 
distance programs, the University employs local site directors. The presentation will cover some of the 
unique aspects used to provide services to distance students. 

This session will benefit professionals in similar settings by presenting a model for the management and 
operation of a student services collaboration between an off-campus representative and the on-campus 
professor. Professionals fi'om other areas might find the methods used by Indiana State University helpful 
in forming a similar program. Distance education professionals faced with the challenge of delivering 
technical, laboratory-based content might find the methods used in the session useful. 

Distance Education at Indiana State University 

Indiana State University (ISU) has a long and distinguished history for offering distance education 
programs in a number of disciplines. Ranging from associate through doctoral programs, the various 
degrees appeal to students interested in business, technology, nursing, and education, among others. 
Indiana State University believes that teaching at a distance is an integral function of university 
instruction. It has, therefore, incorporated distance education into the core functions of the institution and 
has its full-time faculty involved in the development and delivery of distance education programs. The 
distance courses/programs offer students additional opportunities to complete their educational goals. ISU 
is also known by the depth and breadth of the educational programs it provides through distance 
education. 
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BS Degree Completion Program 



Indiana State University has entered into a partnership with Ivy Tech State College and Vincennes 
University to develop seamless program articulations. This is more commonly known as DegreeLink. 
The main intention of DegreeLink is to provide access to a baccalaureate degree for Hoosiers who unable 
to attend a residence campus program due to job, financial, and geographic restrictions. Students 
completing an articulated program can transfer to ISU as a distance education student. Those students 
selecting this program will receive courses through a variety of distance education technologies. The 
Electronics and Computer Technology Department (ECT) is a k^ part of the ISU partnership with Ivy 
Tech State College and Vincennes University. The ECT department offers students the opportunity to 
complete a four-year baccalaureate degree in Electronics technology through the use of distance 
education technologies. 



MSECT Program 

The Master of Science degree program in Electronics and Computer Technology with a major 
concentration in Instrumentation, Systems, and Automation is designed for persons preparing for career 
advancement or improvement. This program prepares students for careers in such areas as electronics 
quality control, technical sales, field representation, biomedical electronics, electrical power, or computer 
control of industrial processes. The degree can be earned through distance delivery to include Internet, 
videotape, and correspondence methods. 

Delivery of Technical Content at a Distance 

The nature of coursework in a technology program includes presentation of theoretical concepts that form 
the basis for the discipline. A major aspect of a technology program is the use of laboratory experiences 
to reinforce these technical concepts. On-campus students are afforded the opportunity to use state of the 
art equipment in high-tech laboratories. Obviously, distance students may not have these same 
opportunities. Some of the methods used to deliver valid laboratory exercises to distance students include 
simulation, lab-in-a-box, and local laboratory utilization. 

Simulation 

Simulation uses a software package purchased by the student or available online. The distance student 
will complete simulated exercises under the guidance of the instructor to meet the objectives of the lab. 
Examples of simulation packages used in the distance programs at ISU include Lab View and Automation 
Studio. 

Lab-in-a-Box 

Pre-packaged laboratory equipment including instructions and data sheets is mailed to each individual 
student. The student completes the laboratory exercise and mails the package back to the instructor for 
evaluation. 

Local Laboratory 

Distance students are provided access to a laboratory near their home for the purpose of completing lab 
exercises. These labs may be located in a technical college, business, or industry. Agreements between 
ISU and the local facility are arranged before the start of the course. 



82 ❖ Management and Delivery 



Local and Remote Management of the Program 



In order to gain and maintain student satisfaction and retention in distance education courses, the 
management of all areas regarding student services related to their education becomes of paramount 
importance. The student support services covered here are based on the initial needs perceived at the 
program’s beginning almost five years ago and the subsequent changes and revisions adapted over time as 
the distance programs progressed. The input needed to incorporate these additions and modifications was 
supplied by student surveys conducted yearly and knowledge gained by everyone involved as the program 
grew. 



Student Support Organizational Structure 

The student support structure in composed of three main groups of people working together for the needs 
of the student. The first one on Indiana State University’s main campus, has been established in the 
Division of Lifelong Learning to deal with inquires, mailings, questions, and/or problems that a distance 
education student might encounter. This department’s interaction with distance students is primarily 
through telephone communications and mailings. 

A second area of student support consists of a network of part time coordinators. These state wide 
coordinators are primarily located on Ivy Tech State College or Vincennes University campuses. They 
serve as a local contact for students. They provide information, answer questions, and refer as needed to 
either one of the three full-time ISU distance education coordinators or the student support group on 
campus. An additional advantage to this group of individuals is that they can interact face to face with 
distance education students. 

The final element of the student support structure includes three full-time Distance Education 
Coordinators. These coordinators have students located in their geographic area referred to them by the 
previous two groups mentioned above. They are able to help the distance student in all areas relating to 
their education. The advantage to the distance student is that these three individuals are knowledgeable in 
all aspects of the University and can meet with them personally to provide information and guidance and 
refer to the appropriate people or departments as needed. Other functions of the Distance Education 
Coordinator are involved in the marketing of the University’s distance education programs so those in 
communities are aware of the education opportunities that exist for them through distance education with 
Indiana State University. 



Student Support Services On the Internet 

In addition to the people involved in the support of our distance students, much information can be 
accessed through the web. Some of these include; 

❖ Online undergraduate and graduate applications for admission 

❖ My ISU portal 

❖ Fee payment 

❖ Registration 

❖ Grades 

❖ Financial Aid information 

❖ Degree Audit Reports 

❖ Schedule of classes 

❖ College transcripts 

❖ Email 



O 

ERIC 



89 



Cockrell & Pearson ❖ 83 



❖ Access to Internet classes 

❖ University information/notices 

* 1 * Complete undergraduate and graduate catalog 

❖ Textbooks 

❖ Distance Learner’s Listserv 

❖ Technical support 

❖ Library Services 

❖ Student Handbook 

❖ Distance Education web sites that includes all distance program information 

Through the cooperation and communication between all parties involved in a distance students’ 
education and the advances in technology, Indiana State University distance education students are 
finding success in their educational endeavors. High levels of satisfaction have been expressed regarding 
ISU programs as gauged by the responses to our annual surv^s. Retention rates continue to climb each 
year. 
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Developing, Archiving, and Disseminating 
Learning Objects: The Process 

Simone Concei 9 ao, PhD 
Instructional Design/Technology Consultant 
University of Wisconsin-Milwaukee 

Rosemary Lehman, PhD 
Senior Outreach/Distance Education Specialist 
University of Wisconsin-Extension 

Background 

In 2001, an American Sign Language (ASL) course was developed by the University of Wisconsin- 
Milwaukee to be taught via the blended technologies of videoconferencing and Blackboard to learners in 
Milwaukee and at The Pyle Center in Madison. This ASL level one course was a pilot project and 
included a deaf instructor, a deaf site coordinator, a deaf evaluator, and other instructional design 
consultants. The course involved one meeting of three hours once a week via videoconferencing during a 
seven-week period, along with instructional activities on the web using Blackboard and a streaming video 
server. In addition to the synchronous and asynchronous nature of the course, 351 video-based “learning 
objects” of the ASL signs were developed and placed on a streaming video server so that the learners 
could practice the signs after each videoconferencing session. Following the course, the learning objects 
were ^ted, reformatted, and tagged for sharing through the Advanced Distributed Learning (ADL) Co 
Lab global knowledge repository. Further, other innovative ways for the learning objects were explored to 
allow increased accessibility and use. This session paper addresses the process and challenges of the 
development of video-based learning objects, their use as Web resources, their revision for handheld 
devices and CD-ROM, and their preparation for sharing through knowledge repositories. 

Video-Based Learning Objects 

“Learning objects” is a term that originated from the object-oriented paradigm of computer science. The 
idea behind object-orientation is that components (“objects”) can be reused in multiple contexts (Wiley, 
2000). For the purpose of our project, we define learning objects as video-based digital entities that are 
available via the Internet where individuals may access and use them simultaneously with a course or 
program, accessible through a CD-ROM for individual use via a computer at the person’s own pace, 
retrievable from a knowledge repository for global sharing, and viewable on a handheld device for 
individual mobile use. 

According to Wiley’s (2000) learning objects taxonomy, the ASL video-based learning objects fall into 
the category of combined-closed type of learning objects because they are single purpose and provide 
users with instruction and practice. Each ASL video-based learning object is an entity unto itself The 
parts that comprise each learning object (text and video images) cannot be separated or they would lose 
their meaning. Without the text the user would not know the meaning of the sign and without the visual 
image, there would be no sign. 

Process and Challenges 

The process of developing, archiving, and disseminating the ASL video-based learning objects involved 
four phases. Each phase utilized a different mode of accessibility for the use of the video-based learning 
objects. 
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Phase One: 
Phase Two: 
Phase Three: 
Phase Four: 



Creation of 351 video-based learning objects for use on the web with the ASL 
pilot project course 

Editing of the video-based learning objects for global sharing and use through 
a repository of knowledge 

Placement of the video-based learning objects on a CD-ROM for the purposes 
of storage as well as learner use when the Internet was not readily accessible 
Revision of the video-based learning objects for mobile use in a handheld 
device 



Phase One: Creation of Video-Based Learning Objects 



During this phase of the process the instructor was videotaped performing the ASL signs. The signs were 
then edited into a video clip format in the following sequence: 



❖ Sign description (text) on black 

❖ A mid-shot of the instructor performing the sign 

❖ A close-up shot of the instructor performing the sign 

❖ Sign description (text) on black repeated 

Signs were grouped into units and then into categories (e.g. nouns, verbs, sentences, etc.). The video 
sequences grouped into categories were saved as Real Media and stored on a Real Media server. A link to 
the Real Media server was created in the Blackboard component of the ASL course for retrieval by 
learners. The ASL learning objects served as a major review and rehearsal component of the course. 

Phase Two: Editing of Video-Based Learning Objects for Global Sharii^ 

During this step of the process, the categories of video-based learning objects created for the ASL course 
were divided and edited into individual signs for database searching purposes. Each learning object 
sequence was formatted in the following order using Adobe Premiere: 

❖ 1 5 seconds of black at the opening 

❖ Sign description (text) on black 

* 1 * A mid-shot of the instructor performing the sign 
* 1 * 15 seconds of black 

❖ A close-up shot of the instructor performing the signs 

❖ Sign description (text) on black repeated 

❖ 1 5 seconds of black at the end 



To ensure that the learning objects had high video quality, were readily accessible from different locations 
and across platforms, and provided rapid retrieval through modem, network, cable, and other high-speed 
connections, a prototype of four video-based formats (e.g., MPEG, Windows Media, RealOne Media, and 
QuickTime) was developed before making them available for the repository. 

An evaluation chart assisted five evaluators in deciding on the effectiveness of the learning objects video 
format for the web-based repository. As a result of the evaluation. Real Media was selected for use. After 
the selection of the video format, all of the video-based learning objects were tagged using the following 
descriptions: 



* 1 * Asset Name: 00 l_test_copyme.mov 

❖ Asset Description 

❖ Content Type: VIDEO 
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❖ Imported By 

❖ Date: Imported: Asset Version: 1 

❖ Is Latest Version: Yes 

❖ Source: Reference 

❖ Source: Phys Type 

❖ Source: Phys Loc 

One of the challenges we found was to decide on the description of the meta tags in order to provide an 
intuitive and accurate depiction of each sign. We tried to maintain the description of the sign to its basic 
meaning. The learning object files were then transferred to the Real Media server. 

Phase Three: Placing the Video-Based Learning Objects on a CD-ROM 

The decision to place the edited video-based learning objects on a CD-ROM was made for several 
reasons: storage, archiving, and retrieval by learners who were not able to access the learning objects 
through the Internet. 

Phase Four: Use of the Video-Based Learning Objects in a Handheld Device 

In order to reach out to a wide variety of users who like to learn in a self-paced and mobile learning 
environment, the video-based ASL learning objects were tested on a handheld device. 

Initially the video-based learning objects were designed to run at 30 frames per second to fit the web and 
CD-ROM formats. It was necessary to reformat the previously used format for the handheld device. 

After testing, the video-based learning objects were reedited from 30 to 1 5 frames per second for memory 
capability. This made it possible to load all 35 1 video-based learning objects into the handheld device. 
During the revision of the learning objects for the handheld device, we inserted several frames of black in 
the beginning and ending of the learning objects and between the two shots of the instructor signing. The 
purpose of these insertions was for clarity — to set off the text and more clearly differentiate between the 
two shots of the instructor and the text that explained the sign. 

An important challenge to address was the selection of the handheld device that would offer the best 
picture and motion quality, and memory capability. Research by technical staff resulted in the selection of 
the Compaq iPAQ 3870 handheld device because of its picture and motion quality, and its extensive 
memory. The learning objects were loaded into the iPAQ using iPAQ software. Further steps and 
challenges, in progress, are devising a way to bulk import the video-based learning objects into 
repositories, populating metadata specifically for the objects, and exploring other issues such as 
compatibility with Management Systems. 



Conclusions 

There is a growing need for the use of learning objects to provide educational resources for faculty that 
enable them to enhance the learning experiences of learners. According to our experience in developing 
the video-based learning objects we realized how flexible and accessible these learning objects could be 
by using a variety of technologies. When we first made the learning objects available, they were limited 
to the use on the website by the learners in the specific ASL course. Because participants of the course 
gave high ratings to the video-based learning objects, we decided to pursue new ways to disseminate and 
share them. 
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The development of the ASL video-based learning objects has heightened our awareness of the need to 
design these objects to fit flexible modes of instruction in order to enhance their accessibility and use for 
learners. This process requires the use of standards, tests, evaluation, and often revision. 

The actual use of the learning objects in the iPAQ has highlighted some new considerations. For example, 
while using the iPAQ 3870 we discovered that there are several accessories that would enhance the use of 
the learning objects: 1) there is an accessory that provides additional memory which will allow us to use 
30 frames per second for the video sequences, providing an even better video quality for the learner; and 
2) another accessory to the iPAQ will enable the learning objects to be projected from the iPAQ to a large 
screen. This will be an excellent way to demonstrate the instructional uses of the iPAQ to a large crowd. 

Because of the value of the ASL learning objects, we are now considering developing ASL video-based 
signs for all five levels of American Sign Language. In addition, we hope that our experience will serve as 
a model for other designers and educators to create learning objects for other content areas. 

(The creation of the ASL video-based learning objects was partially funded by University of Wisconsin 
System and became the collaborative effort of a number of UW institutions.) 
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PJs or Uniforms? 

Reaching Those Who Can’t Reach You: 

Teleseminar vs. Online Education! 

SMSgt Christopher J. Coyne, Director of Education 
Air National Guard Satellite Noncommissioned Officer Academy (ANG SatNCOA) 

Sue Fitzner, Manager, Higher Education Services 
VCampus Corporation 

Introduction 

The topic of “distance learning” is somewhat enigmatic in nature. It brings to many myriad concepts . . . 
all with different applications and different meanings to different audiences. Reaching students who can’t 
be “physically present” in a traditional learning environment (i.e., university classroom, corporate 
headquarters) remains a challenge to both academic and business environments. The purpose of this 
comparison — PJs or uniforms — is to collectively examine and evaluate ideas, methods, analysis brought 
to life with synchronous and asynchronous methods of delivery, while sharing success stories that will 
encourage participants to embrace “learning at a distance.” 

ANG SatNCOA 

The Air National Guard Satellite Noncommissioned Officer Academy (SatNCOA) has provided a 
quality distance-learning alternative for enlisted members since 1994. This program is based on 
sixteen weeks of live, highly interactive instructional television, meeting two evenings a week for 
four-hour blocks. The program concludes with a two-week resident component. The traditional 
resident academy requires six weeks of full-time attendance at the Air National Guard Training & 
Education Center, located on McGhee Tyson Air National Guard Base, Knoxville, TN. This can 
be very disruptive to the civilian employment of traditional Air National Guard members. The 
SatNCOA offers an appealing alternative, which not only provides an outstanding academic 
experience, but also bolsters recruiting, retention and unit morale. 

VCampus 

VCampus Corporation is the leading provider of end-to-end e-Leaming solutions. The company develops, 
manages and hosts turnkey, web-based learning solutions for corporations, government agencies and 
higher education institutions. VCampus enables these organizations to offer complete global distance 
learning solutions to their customers, employees, distributors, suppliers and students. VCampus’ e- 
Leaming solutions are designed to help clients deliver higher education programs to adult students; 
improve the performance of their distribution channels and suppliers; measure and develop their 
employees’ knowledge, skills and abilities; and increase their customers’ satisfaction and loyalty. 

Comparison 

The SatNCOA, however, offers a slightly different perspective to those interested in designing a course 
with the magnitude of VCampus. Rather than employing the typical “talking-head” syndrome used by so 
many programs, the SatNCOA utilizes a “team-teach” approach (very similar to a typical Sports Center 
broadcast), with a duo of professionally trained, dynamic instructors who concentrate on delivering 
curriculum through this iimovative medium. Their tools are a combination of superior on-screen 
graphics, video clip examples, a large plasma screen between the instructors used for multi-media effect. 
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and a behind-the scenes team of directors, producers and graphic artists. Although the SatNCOA’s 
primary customers are students in an academic setting, the road paved by this program can be duplicated 
in many other training settings, regardless of goal or objective. The SatNCOA has attempted to exercise 
the “talking head, non-rehejirsal, untrained talking-head demons” of the world and replac^ them with 
highly skilled teachers who imderstand the difference between the traditional and distance learning 
classrooms. 

VCampus Corporation distributes a courseware library of more than 5,000 web-based courses and has 
delivered more than 2.3 million courses to over 640,000 adult learners. Headquartered in Reston, VA, 
VCampus is a completely outsourced, remotely hosted and totally scalable distance education and training 
delivery solution. The services offered enable clients to easily manage the enrollment, registration, 
tracking, testing, grading, administration and certification of distance learners. Over the years, numerous 
other distance-learning methods have come to life, including computer-based training (asynchronous and 
synchronous) and on-line web-based curriculum designed to serve the needs of nontraditional students in 
all walks of life. Client feedback and evaluation come from a variety of partners to include the General 
Services Administration, Park University, University of the Incarnate Word, State Farm Insurance, 
Security College Online and more. 

Join us to examine these programs that offer two environments and prospective, as we critique, evaluate 
and discuss a variety of topics throughout this presentation; 

* 1 * Recruitment of instructors 

❖ Technology and Equipment 

❖ Support Services 

❖ Diversity 

❖ Quality and Academic Excellence 

❖ Learning Community and Environments 

❖ Faculty prospective and expectations 

❖ Student learning outcomes 

❖ Administrative prospective and functions 

❖ Asynchronous vs. Synchronous 

❖ Curriculums 

❖ Growth Patterns 

❖ Explore new partnerships 

❖ Increased access and opportunities for its students 

❖ Cost avoidance for their units, and quality professional education 

❖ Advantages and Disadvantages form three different prospective: 

❖ Student 

❖ Faculty 

❖ Administration 

❖ Time Zones 

❖ Family Need 

♦♦♦ Job/Professional commitments 
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Design Issues, Standards, and Technology: 

Can Online Assessments Measure More Than Recall? 



Laura E. Culver, M. Ed. 
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Instructional Design Group, LLC 



Good Assessment Question Ingredients 



During the instructional design process, much time and attention is paid to conducting in-depth analyses, 
developing sound learning objectives, creating stimulating training materials and job aids, and then 
quickly throwing together some assessment questions designed to verify that learning took place. These 
questions, developed just before the project due date often consist of questions that are too obvious, too 
obtuse, and unfortunately, too often do not measure the intended learning outcomes. Sound familiar? 

Generally, designers understand the purpose of assessments and agree that they are an important part of 
the design process. Although an intellectual understanding of the importance of assessments exists, often, 
the requisite skills needed to develop solid assessment questions is lacking. Instructional designers (in the 
corporate sector) come from varied backgrounds, and may have little or no formal training in learning and 
testing design. In addition to this challenge, many software development packages used to develop online 
learning severely limit designers’ ability to create questions that can test to higher learning objectives. 
Because some development tools are very restrictive, the objectives are positioned at the 
knowledge/recollection level of the cognitive domain to ensure that assessment questions measure the 
desired learning outcome. In other words, the development tool dictates how learning will occur and at 
what cognitive level. These issues combine to produce assessment questions that are low on the cognitive 
domain and not truly indicative of whether meaningful learning was achieved. 

Because of technology’s influence, online assessment questions rely heavily on true/false and multiple 
choice as the models of choice. Both are valid assessment methods; however, the problem typically lies in 
the construction of poorly worded questions and distracters rather than the model choice. This results in 
assessment questions and answers that are of little value. For those who use the results of these 
assessments to determine training effectiveness quantitatively, accurate interpretation is dismal at best. 
Although technology capability does influence the design of assessment questions, fundamentally, well- 
written, measurable objectives and good assessment design create the foundation for meaningful, valid 
questions. 

Taking technology constrictions out of the assessment development recipe, well- written objectives that 
are measurable in an online environment are the first essential ingredient. Unlike face-to-face training 
objectives, online training objectives are designed to measure outcomes not observe outcomes. 
Understanding the differences between measurable and observable outcomes when developing online 
training is critical. Designers and reviewers need to ask “what will learners be able to do when they have 
completed the course?” to ensure measurable learning outcomes. Depending on the online topic, it is 
possible to achieve measurable objectives within the application and analysis realm. When online training 
is coupled with a face-to-face training session using the concepts from the online session, higher cognitive 
levels can be achieved because the element of observation is available to the facilitator. 

The second ingredient of the assessment development recipe consists of setting standards. Standards 
enable designers to verify that each question and distracter matches the intended learning outcomes, and 
fairly and accurately evaluates course learning. By establishing standards, the consciousness of the 
developers and reviewers is raised, and questions and distracters are more carefully scrutinized. Setting 
standards positively influences consistency, ensures that what is taught is tested and improves the quality 
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of questions designed. Further, standards help reduce the use of obtuse or obvious distracters and aids 
designers in focusing on the truly relevant, core learning they intend teaching. Finally, standards can act 
as a catalyst for helping designers develop “fair” assessments. After all, the purpose of an assessment is to 
give learners the opportunity to demonstrate what they know (Suskie, 2000). 

The last ingredient calls for training instructional designers and reviewers. The training needs to cover 
both the design of learning objectives and how to construct well-worded questions and distracters that 
match the intending learning outcomes. Designers need to be able to determine the cognitive level of an 
objective and then review assessment questions to determine if there is a match between learning 
intention and outcome. Once they feel comfortable determining this, given a set of standards, designers 
then review a series of questions and distracters to evaluate their effectiveness or ineffectiveness and 
suggest ways to improve them. When designers can identify what constitutes a good question and 
distracter, can determine the cognitive level of the objective and verify that the objective and assessment 
question match, conscientious application of this knowledge to assessment writing will result in better 
questions that more closely match intended learning outcomes. 

Online Learning Assessment Question Types 

Online assessment question types are closely tied to the technology capabilities of the development tool. 
Although there are exceptions, most online assessment questions are confined to three model types: 
multiple choice, multiple response (matching) and true/false. Each of these models has strengths and 
limitations. It is up to designers and reviewers to overcome these limitations until technology capabilities 
and accessibility are a non-issue. 

Multiple choice assessments can be designed to measure higher order learning objectives by presenting 
practical or real world situations. A short paragraph can be used to describe the problem. Learners are 
then asked to review the listed items to assess the application of principles or evaluate alternative 
procedures (Kehoe, 1995, Poland 1999). 

Multiple response assessment questions are useful for testing basic knowledge and are a variation of 
multiple choice questions. Multiple response questions have the reputation of being easy because learners 
can use the process of elimination to determine answers. Using an item more than once, or not at all, 
reduces the process of elimination tactic. To use this technique successfully, designers need to write very 
clear directions (Poland, 1999). 

True and false assessment questions are overused and typically poorly developed. True and false 
statements are generally perceived as being easier to write, when in fact, good true and false assessment 
statements are difficult to write. In an effort to develop an accurate statement, designers carefully word it, 
which increases the probability that learners can guess the correct response. Typically, development 
models confine true and false statements to two questions per interaction. This further increases the 
likelihood of guessing the correct response and decreases ^e effectiveness of the assessment (Poland, 
1999). 



Assessment Question Standards 

The purpose of standards is to provide designers with a series of guidelines they can use to determine if 
their questions and distracters match what was designed in the online course. Setting standards creates a 
performance expectation level and gives both designers and reviewers a benchmark against which to 
judge questions and distracters. These guidelines may include general standards, direction text standards, 
media/graphic standards, question/statement standards and distracter standards. 
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General standards may include guidelines such as; 

❖ Assessments must test to the instructional objectives. 

❖ Identify one content point to be tested. Do not introduce new material or test skills not directly 
taught. 

❖ Focus assessment questions on key concepts, principles and skills that are essential. Avoid 
questions that only require recall of isolated facts. 

Direction text standards and media/graphic standards may include guidelines such as: 

♦> If there’s one correct answer, “click (or drag) the statement/answer...” 

❖ Use text-only models rather than audio models to present question information, unless the 
content dictates that audio information be present^ to clarify the question. For example, a 
training session about telephone techniques should use audio models so that learners can hear 
tonal inflections to determine the best response. 

❖ When re-using graphics from a lesson, ensure that the graphic provides enough information to 
be meaningful without making the answer obvious. 

Question/statement standards and distracter standards may include guidelines such as: 

❖ Construct questions/statements as either incomplete statements or direct questions that include 
noun/verb phrases. Exclude superfluous information as a way to introduce a question. 

❖ Design questions so that learners are asked to choose what is rather than what is not. Do not use 
always or never in the question/statement. 

❖ Arrange distracters in ascending or descending order of length. This prevents learners from 
immediately assuming that the longest answer is the correct answer. 

Standards are based on design principles, not the technology being used to develop online learning. By 
taking technology out of the design equation, assessment questions - including true and false - become an 
effective tool to determine whether learning occurred. 

Modifying Assessment Development Models 

When technology is effectively used, it becomes almost imperceptible to learners. When easy to use 
development models are combined with quality content, it results in learners concentrating on what they 
are learning rather than how it is being presented on the screen. 

Major revisions to online training development software are not necessarily needed to effect desired 
changes. For example, true and false statements are typically written as a single statement and represent 
one screen of online learning. This gives learners a 1 :2 probability of getting the correct answer. 

However, if a total of four statements are used, now learners have a 1 :4 probability of getting the correct 
answer. 

To increase the effectiveness of multiple response questions, using models that include options for an 
even or odd number of response items allows designers to craft questions that either do not use a response 
or re-use a response to reduce the use of the process of elimination tactic. This is not suggested as a 
means of tricking or confusing learners. The goal of assessments is to determine whether or not learning 
occurred, not that learners are good at taking tests. 

Implications for Designers and Reviewers 

Writing good assessment questions is difficult, and because many designers lack the skills needed to 
develop effective questions, this odious task is often left until the very last moment in the product 
development timeline. The designers then hurriedly develop questions and distracters that may or may not 
measure the intended outcome, and reviewers are rushed in an attempt to read and edit the questions and 
distracters to meet the deadline. Delaying the development of questions and the subsequent review of 
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assessment questions is not intentional. It is human nature to put off those things we despise doing. 
However, it is to the detriment of the product when the delays occur because designers do not know how 
to craft effective questions. 

Fundamentally, good assessment questions are the result of good design first and technology use second. 
Choosing the appropriate type of question, multiple choice, true and false or multiple response directly 
correlates to the desired assessment outcome. Are learners being asked to recall facts or analyze and apply 
information? Is the content introductory or more advanced? Do the assessment questions align with the 
objectives and are they observable in a web-based learning environment? The implications of not asking 
these questions often result in the lack luster, ineffective assessment questions that abound in online 
training. 
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A Statewide Brokerage House for Online Education: 
An Education Business for Higher Ed. and K-12? 

Andrea Dean, Educational Technology Liaison 
WiscNet 

Dave Lois, Executive Director 
WiscNet. 

Joan Peebles, Coordinator of Technology & Learning 
Madison Metropolitan School District 

Hal Schlais, Director for Learning Technology Development 
University of Wisconsin 

Ross Wilson, Technology Services Director 
Cooperative Education Services Center 10 



There is a current and growing need for a statewide brokerage house for education. Have you found 
yourself asking questions like these? 

❖ “We have five students that need AP Calculus. We’ve looked for a part-time teacher, but just 
can’t find one. What are some other options?” 

❖ “We are looking for virtual field trips to support our middle school science curriculum. Where 
can we go?” 

❖ “We want to find professional development opportunities having to do with using an online 
environment to support teaching and learning. What’s out there for us?" 

Imagine being able to pose these questions and receive an array of options from other educational 
resources in the state such as: 

❖ K12s 

❖ Higher education institutions 

❖ Technical colleges 

❖ Libraries 

❖ State and other agencies 

If something isn’t prepackaged, there may be someone willing to create it based on your need. What you 
need is a connection. 

A statewide education brokerage house may be one such connection. It’s purpose would be to create a 
forum for education entities to provide and receive content. It wouldn’t be about making money. Content 
could be free in some instances, and not in others (i.e., tuition, for courses). It would be an environment 
that would bridge disparate issues and jargon and match a receiver’s requirements with a provider’s 
content, ideas and potential. This brokerage house would be a conduit to complete the transaction within 
the limits of each entity’s business model. This concept is not a searchable database of quality content as 
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with IDEAS, Merlot, or MarcoPolo*. It is a place where people could exchange, broker, purchase, or 
consume educational content. It could incorporate existing content, and facilitate content “made-to- 
order.” 

Is there is a need for an education brokerage house? Yes. 

There have long been both producers and consumers of educational content. The perpetual 
challenge has been for each to find the other. 

Is the need growing? Yes. 

National trends indicate that online teaching and learning in the K12 environment is on the rise. 

* 1 * 1 2 states have established virtual high schools (five more are under development) 

❖ 32 states have an e-leaming initiative 

❖ 25 states have laws that allow online charter schools to be established (to date approx. 30 
online schools have been established in 12 of these states) 

❖ 50 states allow districts to sponsor e-leaming initiatives 

* 1 * 1 3 states regulate non-state sponsored learning initiatives 

❖ Several thousand high school students in 23 different states are taking online courses in 
public schools 

As this need continues to grow, where will these folks find quality content without having to develop it 
from scratch? An education brokerage house addresses this need by creating an environment for 
education to serve education. It would create a fomm for education entities to provide and receive 
content. Suppliers and consumers need to find each other, they need a mechanism for compensation, and 
they need two-way communication that bridges issues and jargon unique to each institution. This isn’t a 
traditional market model where one party “guesses” at the other’s needs, then attempts to “sell” it to them. 
The forum bridges the receiver’s requirements and ideas with provider’s content, ideas and potential. 
Essentially, all parties need to be equal; whether you are the consumer or provider, or from higher 
education or K-12. 



Examples 

Officials from Madison Metropolitan School District in Madison, Wisconsin, are currently investigating 
the viability of developing their own online Algebra 1 course because of the difficulty in finding quality 
content to purchase. They are intrigued by the concept of creating their own course, seeing it reused 
within their own and other districts and perhaps even recouping some of the development costs. 

Cooperative Education Service Area 1 0, (one of twelve Wisconsin Department of Public Instruction 
regional education service centers), currently brokers approximately $500,000 worth of content across its 
regional distance education network. 



' IDEAS - www.ideas.wisconsin.edu, provides Wisconsin educators access to web-based resources for curricula, 
content, lesson plans, professional development and other selected resources. 

Merlot - www.merlot.org, a free and open resource for higher education. Links to online learning materials are 
collected here along with annotations such as peer reviews and assignments. 

MarcoPolo - marcopolo.wordcom.com, provides no-cost, standards-based Internet content for the K-12 teacher and 
classroom. 
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Each fall, K12 school officials and administrators, guidance counselors, representatives from the 
University of Wisconsin Eau Claire and others gather to discuss what content should be offered to 
students for coming year. They state their needs and offerings and out of that conversation, create a 
booklet of offerings from which students can register to attend courses at a distance. CESA 10 serves as 
the administrative and billing agent for this process. 



This is a working model for the education brokerage house concept, however, to do this state-wide, it 
needs to be a model that will scale to that degree. It needs to be able to grow, change, evolve and adapt to 
the community it serves. 

It is important to remember that this concept is not a replacement for the existing searchable databases, 
repositories, referatories and the like. It is the next evolution or perhaps, next generation of those models. 
The concept is in its early stages and is still in development. It will require participation from all of 
education to make this concept work. Those that create content, those that consume it, those that support 
the educational process and infrastructure. It cannot be done by one entity alone. It will be critical for all 
of education to be involved to ensure a successful brokerage house. A working model would truly have 
education serving education. 
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How Should We Talk? 

Scaffolding the Work Process for Online Groups 

Vanessa Paz Dennen, Ph.D. 

Assistant Professor, San Diego State University 

The familiar sound of student groans when a group project is announced may not be audible in the online 
class, but it remains present. Having students work in online groups is as simple as telling students they 
will work together. There are the regular challenges of group projects, such as ensuring a group is getting 
work done, getting along, distributing work equitably, and learning in the process. Each of these is a bit 
different in an online setting. Additional issues to be dealt with include finding ways of helping students 
meet and collaborate online, and helping them structure their work process. 

From the student perspective, working in an online group poses many new challenges. They may be 
familiar with scheduling live meetings and discussing ideas in a face-to-face environment, but doing the 
same online is likely a new experience. Rather than let students flounder and figure it out on their own, 
which often results in fhistration with the assignment and group members, instructors can provide 
structures that help students be more successful with their process, meeting not only the specific content- 
based learning objectives but also helping the learners become better online communicators and 
collaborators for future projects and work experiences. 

Putting Together Groups 

When grouping students for projects, it is important to take into consideration the likelihood that a group 
will be able to work together successfully to complete the task. This means that the group needs to have 
shared interests, adequate abilities across the members to complete the task, and the ability to 
communicate effectively with each other. These are not really any different than the concerns that face an 
instructor forming groups for a classroom-based course, but teasing out this kind of information about 
students may be different. 

One idea an instructor might try includes having students discuss potential project ideas for a week before 
forming groups, which may help some students find others with common interests and begin the group 
formation process. I would recommend, however, not relying on this method solely for group formation 
as there will always be some students who do not automatically fall into place with a group. A brief 
survey — easily created using most online courseware; otherwise possible via e-mail — can be used to 
gather information about students interests as well as their preferred communication and collaboration 
modes (synchronous or asynchronous), writing styles (collaborative or divide and conquer) and 
scheduling issues (early bird or last minute writer; constraints that might preclude participation at a 
certain time). Armed with this information along with what the instructor knows about the students from 
their earlier class participation, groups that are likely to work well together can be formed. 

Structuring the Assignment 

At first glance, a group project in a classroom-based course might have little structure beyond a start date, 
an end date, and an end product. Because the students see each other regularly in class and the instructor 
may even allow some in-class time for project work, that may be all the imposed structure that is needed. 
In an online scenario, however, the impact of not seeing each other regularly can cause great 
procrastination amongst students, resulting in fioistration for some and the inability to contribute to the 
group project for the others. 
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Requiring that groups generate and submit a work plan can be very empowering. It helps the group both 
plan its own path and lets the instructor know whether or not the group is setting itself up for success. 
Such a plan might consist of a proposal that outlines the project concept, a schedule, and a listing of what 
responsibilities each group member will undertake and a statement of the group’s communication 
commitment. A model of a work plan may be provided to help students determine the types of agreements 
they need to make prior to working together. 

Should the instructor not want students to structure their own work processes, they can build in specific, 
incremental due dates and deliverables for students. Breaking the project into smaller parts that are due at 
specific times helps keep students focused and requires them to check in with each other regularly. The 
instructor need not heavily assess these smaller parts, as that might be overwhelming, but they should be a 
required component of the project. One idea for ensuring students get feedback at such points is to assign 
other groups to provide peer responses. Everyone benefits from this situation, with each group receiving 
some feedback and getting the chance to see what others are doing and how they have interpreted the 
assignment. 



Determining Communication Tools 

There are many ways that online groups can collaborate and communicate. A tool such as Blackboard 
provides group spaces with discussion boards, chat, a file exchange and group e-mail addresses. Outside 
of these courseware tools, students might decide to set up their own e-mail lists, use free groupware such 
as yahoo groups, or use the telephone. In some instances they may even be able to set up face-to-face 
meetings. Planning modes of communication is really important; groups who do not plan can fall prey to 
the group member who does not check e-mail regularly or who does not know where group files are 
posted. Decisions need to be made regarding what tools will be used for what purpose and how frequently 
members are expected to check in via any asynchronous technologies. 

Some successful groups set up a weekly time for synchronous communication via chat. While 
asynchronous tools seem to be more efficient for distributing documents and getting group commentary, 
synchronous tools can be useful for making decisions and assessing progress. Synchronous meetings 
typically go better if there is an agenda and if all members come prepared. Instructors can help students 
communicate effectively with their groups by providing these kinds of tips in advance. 

Checking on Progress 

Having students submit regular updates to the instructor — as a group or as individuals — is a useful way 
of ensuring that students stick to the structure that has been decided up on for the project. If progress 
reports are required for the group, a project manager may be designated with the responsibility of writing 
and sending the update. If they are required at the individual level, it becomes possible to get perspectives 
on how well the group is working together and determine whether or not any help or interventions are 
necessary. To make it easier on both the students and instructor, a web-based form can be used for the 
updates, asking students to answer particular questions based on what they should be working on at that 
time. This helps keep all responses in an ordered place and ensures the important topics are covered. 

Assessing Process As Well As Product 

These ideas for structuring group work make students focus heavily on their work process. A process- 
focused approach is likely to result in a quality product, but nonetheless it can be important to assess the 
students’ process as well as the final documents that they turn in. Doing so often makes students more 
likely to undertake good work habits and reflect on the process that helped them complete their group 
work. It also helps the instructor learn more about how students approach that particular assignment so 
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that the deliverables and the overall process can be better structured and articulated the next time the 
course is offered. 



Closing Thoughts 

These are some general ideas for how to help students be successful when working in online groups. The 
most important concepts include herein are the following: 

❖ Students need help forming groups in an online class 

❖ Students need help determining how to approach a group project, including how to chunk and 
schedule their work 

❖ Students need help to learn what communication tools their groups should use for particular 
tasks. 

❖ Progress reports are a useful way of keeping the instructor informed of group progress (and 
encouraging the group to remain focused and on schedule) 

❖ Assessing student process as well as product can be a useful way of encouraging good student 
work habits and gathering formative evaluation data about the integrity and effectiveness of the 
assignment 

These ideas for helping students are general in nature; each instructor needs to find the amount and type 
of structure that is most appropriate given the content, the assignment, the level and characteristics of the 
student and the instructor’s own personal style and preferences. Above all else, most important is being 
sensitive to the students’ need for assistance in approaching the process of online group work as well as 
for learning the material covered by the course. 
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Digital Delivery to the Desktop: 

Documents Anytime, Anywhere 

Ulrike Dieterle 

Head, Access and Delivery Services 
Health Sciences Libraries 
University of Wisconsin-Madison 

The Health Sciences Libraries, University of Wisconsin-Madison, are offering faculty, staff and 
graduate students an attractive alternative to traditional forms of document delivery. Based on 
high-speed transfer, Web-based interface and order tracking. Library Express combines recent 
technologies with a strong customer service orientation to provide easy and convenient access to 
local and worldwide collections 

On a Big Ten campus spread out over 933 square acres, distance can pose a significant barrier to 
retrieving print resources from over 40 libraries. For those working off campus in the many research 
stations and clinics, students completing rotations and clerkships in other parts of the state or the nation, 
the challenges of access to health sciences materials become even more daunting. At the University of 
Wisconsin-Madison, a new document delivery service. Library Express, is bridging the distances and 
providing easy and affordable access to articles, book chapters and tables of contents from local library 
collections and beyond. 

Library Express users can initiate their requests through a Web-based interface, track the status of their 
requests online and retrieve digital documents in PDF format from their networked computer of choice 
anytime, anywhere in the world. Four campus libraries currently provide this service to a potential 
combined user population of approximately 24,000 faculty, staff and graduate students. \^ile the service 
is currently not open to undergraduates, exceptions are generally made for distance learners of any status. 
Libraries and their Library Express users are matched by subject area affiliation, major and, when 
necessary, by manual entry into the system. Authentication and access are determined by an active 1 1- 
digit campus ID. Within the Health Sciences Libraries (HSL) Library Express development occurred in 
three basic phases - an extensive planning phase, a carefully staged implementation phase and the final, 
fully operational phase with second generation software. The last phase opened the service to all faculty, 
staff and graduate students at the University of Wisconsin-Madison. 

The HSL consist of three separate library facilities (main health sciences, pharmacy and hospital 
collections) which serve a diverse campus population including the Medical School, School of Pharmacy, 
School of Nursing, basic sciences. History of Medicine and other health sciences and related fields. User 
groups are very mobile, crisscrossing not only a large campus but also traveling the state, the nation and 
the world. 

The HSL have a long tradition of making library materials readily accessible to primary user groups. For 
many years library users have had access to a shuttle van that stops twice a day at each of the three 
libraries to pick up and delivery library materials. Requested items are shuttled for convenient pick up at 
the site chosen by the user. This service provides one-day turnaround and is, for some, still the document 
delivery method of choice. While very efficient in getting the materials to the users in a timely fashion, 
the service has one major disadvantage. Physical items being transported are out of circulation for days 
and not available to others at the home library. With an increasing number of electronic journals available 
and readily accessible from our Web pages, HSL users are quickly adopting the idea of digital retrieval as 
a viable option. To broaden the scope of easy access to print-only titles/volumes, digital delivery to the 
desktop seemed the next logical step and a welcome replacement for other more manual and cumbersome 
methods of the past. Library Express also offers the opportunity to provide one point of access for 
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multiple types of delivery services running in the background. When fully implemented, users will no 
longer have to ponder where to go and which forms to fill out for intra-campus or inter-campus services. 

With the advent of appropriate and easy to use technologies, more comfort with computers and an 
articulated commitment to serving the information needs of our library users in a timely and convenient 
manner, the implementation of digital delivery to the desktop was given top priority. After 2 Vi years of 
operation, the service has not yet reached its capacity and continues to grow. The current manifestation of 
Library Express is viewed as a bridge to the electronic library of the future, as a road to the next 
generation of technology-enabled library services. 

Planning for Success 

Heeding lessons learned earlier in 1999 from a premature start and a quick retraction of the service in the 
spring of that year, the HSL put the smooth implementation of Library Express high on the list of the HSL 
priorities. The newly hired Head of Access Services was given the mandate to bring up a new digital 
delivery program in a timely and organized manner. The general components of the service would be 
based on an existing document delivery pilot already in place at another campus library. Early in the first 
planning process. Ophthalmology, a department of approximately 60- plus members, had expressed a 
desire and need for electronic desktop delivery to both on and off campus locations. Members of this 
department are widely dispersed and have a broad range of information needs, including clinical, 
instructional and research support. They had also agreed to work with HSL to get the bugs out of the 
program during the beta phase. The challenges posed by this department made it an ideal test group for 
distance delivery services. 

At this point, the project management team consisted of a newly hired .5 FTE dedicated to the Library 
Express planning process, the Head of Access Services and coordinator of the hospital library who had 
experience with the first roll-out. 

Clearly defining service parameters and setting service standards early in the planning phase, kept us on 
track, outlined basic expectations, reduced misunderstandings and streamlined staff training. HSL Library 
Express staff agreed to do everything humanly possible to make access to the service easy and effortless 
for our user groups who repeatedly listed a lack of time as one of their most pressing dilemmas to 
information retrieval. Our goal was to deliver articles from all three health sciences collections within 1 -2 
working days, 24/7. Articles would be ordered through a uncluttered Web-based interface, received by 
processing staff in email format, materials retrieved from the library shelves, scanned and delivered to the 
user with a link pointing to the PDF document sitting on the server. Adobe Acrobat would be used for 
scanning materials. Data storage and report generation would be MS Access-based. The main health 
sciences library, which has the largest collection of the three, would serve as the recipient and screener of 
all incoming requests. Requests for materials located at either of the other health sciences facilities would 
be electronically forwarded for processing and delivered directly to desktops. Requests for items from 
non-HSL libraries would be retrieved manually from other campus libraries whenever possible, returned 
and scanned for delivery to the user. Requests not found on campus would be sent to our HSL interlibrary 
loan staff for processing and then retum«d to Library Express staff for scanning and digital delivery. All 
requests would be delivered electronically. No paper delivery options would be available through Library 
Express. Links to scanned materials would be available on our server three weeks, then automatically 
deleted from the server. Color images would be delivered whenever requested and available. Status of 
requests throughout the processing queue would be posted to the users email box using canned messages 
manually generated by staff. No fees were charged during the beta phase in exchange for input from and 
the patience of our users. In summary, it was a very labor-intensive process for library staff. 
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Setting a realistic implementation timeline was of paramount importance to the project management team. 
Keeping a vigilant eye on the details helped us stay grounded and focused on the ultimate goal to provide 
a smooth and speedy document delivery experience for the user. A loose timeline was set at “as soon as 
possible, but not before we are truly ready.” After one false start and a project cessation lasting 6 months, 
we needed a successful outcome to restore confidence and sustain momentum. A successful jumpstart 
was perceived as imperative to the extended well-being of the program. 

During the fall and winter of 1 999/2000 staffmg and equipment demands were closely reevaluated. The 
library director made it clear that staffmg as well as other forms of program support such as space 
requirements and equipment would be promptly provided. The .5 FTE hired in the fall to assist with 
initial project planning was no longer sufficient to handle projected development activities, staff training, 
setup, promotional and processing duties. The .5 was converted to a full-time position with a robust 
position description. Identical scanners (Fujitsu ScanPartner 600C) and software (Adobe Acrobat 3.0) 
were purchased and set up at each of the three health sciences libraries to assure uniform processing and 
training at each site when demand picked up. 

Since one Library Express shop (Wendt Engineering Library) was already functional on the campus, the 
first few months of planning were devoted to familiarizing ourselves with the successful processes already 
in place and how we could adapt these to our HSL environment. We were able to emulate much of the 
workflow and “borrow” the home-grown software that allowed us to scan, mount and deliver image files 
with less effort. Library Express staff was trained to verify citations in a number of databases and to work 
with email clients, MS Access and Adobe Acrobat. Student assistants were hired to retrieve, scan and 
post articles to the server. Trouble-shooting and communication with users was limited to Library 
Express staff, closely monitored and carefully documented. 

During this time a detailed procedures manual was designed to support training and to communicate best 
practices. A Web page was developed to provide a point of access and source of service information for 
our users. Promotional materials and other aids, such as a letters of introduction and a service brochure 
were also developed. Other functions and practices of the first generation system were worked out piece 
by piece in countless meetings and discussions throughout the 16-month beta phase. 

Implementation 

Starting out small afforded the project management team time to assess, tweak and fine-tune the service 
before moving on to serve a larger and more complex user group. We introduced the initial version of 
Library Express to members of the Ophthalmology Department (clinicians, faculty, residents, fellows and 
support staff) during February 2000 and opened the service to them March 1 . Library Express staff 
conducted on-site demos, circulated letters of introduction, trained Ophthalmology staff in the efficient 
use of the service and closely monitored their progress. A Web-based tutorial was designed to provide an 
introduction to new users and refresh the memories of the occasional visitor. Technical issues were 
addressed immediately, often with an on-site visit to see first-hand what obstacles the users were 
encountering. Starting with a small group (approximately 60) with varying needs and varying degrees of 
computer comfort levels, allowed for more control, better communication and more immediate resolution 
of problems as they surfaced. 

After working with Ophthalmology for two months, we added the School of Pharmacy to the Library 
Express test pool. The pharmacy graduate students doing clerkships throughout the state posed challenges 
- how to deliver large PDF files to users with minimal hardware configurations, through firewalls and 
over slow modem connections. Additional user groups were added incrementally throughout the year by 
invitation only. Promotion was very focused and controlled to keep activity levels in line with our ability 
to provide a high level of service. Staggered additions to the Library Express service roster included the 
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Department of Surgery, Student Health Services, Psychiatry and the School of Nursing each bringing with 
them unique demands and service challenges. Contacts were identified for each new group and asked to 
contribute to the on-site promotion and local trouble-shooting. 

Although Library Express staff had been meeting often, we now felt a need to formalize the meeting 
calendar in order to include staff from all three libraries who were processing more requests. Staff hours 
during this time tripled as requests increased ten-fold. The first month of the beta period with only 
Ophthalmology on board, we received 129 requests. Sbcteen months later during the last month of the 
free beta phase, we received 1881 requests. Business was booming. 

The Library Express working group had now grown to include the Head of Access Services, the 
coordinator of the hospital library and front-line Library Express staff (1.5 FTE). Regular weekly 
meetings were helpful in keeping us on track, addressing problems quickly, making corrections 
immediately and communicating any changes in policies and procedures to all Library Express staff at all 
three processing locations. Regular meetings continue to this day, but less frequently. 

Throughout the beta phase the Library Express team remained flexible yet focused, smoothing out the 
bumps in the road as they came up, willing to make changes, if needed, to improve the quality of service. 
Nothing was set in stone. Everything was up for evaluation. Attention to detail became our mantra. Staff 
scrutinized, assessed and evaluated each step of the process. We conducted a time study which tied 
staffing needs to volume processed. Daily, weekly and monthly statistical reports generated from MS 
Access were helpful in identifying trends (or the lack of them) and aided with scheduling, estimating 
equipment and space needs as well as planning for future service enhancements. 

During the beta phase we also listened carefully to our users. Two questionnaires were sent out, one in 
May 2000 to approximately 100 users and the second in February 2001 to 350 users of Library Express. 
Beyond a high degree of satisfaction with the service, responses to the early survey also reveal^ a desire 
for color images, that faculty were doing their own requesting and that we needed to do a better job 
promoting the service. Responses to the February 2001 survey indicated that the suggested fee of $1 per 
delivered article would substantially reduce the number of requests. A majority of respondents clearly 
stated that they would exercise more caution when ordering to avoid the $1 fee. This was very useful in 
planning for short-term staffing needs. Other information uncovered from this survey included a need for 
24-hour service, a desire to have tables of contents delivered and a confirmation that most Library Express 
users loved the service and found it extremely helpful in conducting both research and their daily 
business. 

One of the most rewarding experiences during the beta phase was working with colleagues across campus 
to design generation two software and to open Library Express to all faculty, staff and graduate students 
served by four different libraries each with its own specific user group. During this time of collaborative 
development, the group made a wish list of Library Express features that each site wanted in place in the 
new version, both on the public/user side and on the processing/staff side. Out of these meetings came an 
end product that had a common look and feel, yet one that was able to retain some of the site-specific 
features and functions. For example, HSL had for most of the beta offered the option of color images. 

This was an important feature for our Ophthalmology group and others who needed the color variations of 
photos. Blacks, whites and grays were not always as useful. This feature choice was incorporated into 
the design for the HSL Web-pages, but not for other shops. The new software would be totally Web- 
based, eliminating the awkward interactions between email clients, home-grown programming, MS 
Access and other bits and pieces that had been cobbled together. 

Working together with campus-wide Library Express staff also allowed us to share general document 
delivery experiences and philosophies. It strengthened our understanding of current practices in other 
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document delivery areas on campus and created opportunities for further collaboration and partnerships 
beyond Library Express. 



Library Express: The Next Generation 

After testing the new software with a small sampling of current users, the second generation of Library 
Express software was activated July 2001 . Bugs and potential enhancements were identified and 
scheduled for resolution. None prevented requesting or delivery, but did add to confusion on the staff 
side. We all experienced some growing pains. Reports were not in place when the new system was 
activated, resulting in continued data entry with MS Access. User and transaction data, now collected by 
a Web-based system, could not be harvested without considerable programming, which at this writing, is 
still in progress. 

In anticipation of the $1 per article fee, users swamped us in the free periods of May and June resulting in 
our highest productivity to date — 1513 and 1881 respectively. During the month of July, our activity 
level fell to less than half confirming what we had learned from our survey earlier in the year. Since July 
the number of monthly requests has continued to climb, albeit slowly, to leveling out at approximately 
1000. 

The new Library Express software provides a total Web-based environment for both users and processing 
staff. The only remnants of the previous system are the manual scanning process which we have found no 
way to streamline and the initial citation verification and location procedures. Users now have the ability 
to track the progress of their requests online limiting the amount of email generated in communicating 
request status with users. A built-in request forwarding feature allows requests to be moved to other 
Library Express shops for retrieval and processing. We no longer send students to other campus libraries 
to copy articles found there. This significantly reduces the turnaround time for those deliveries. 

As the news of this new service spreads across campus. Library Express is rapidly replacing 
older, less automated levels of document retrieval and delivery. This successful program is a 
direct result of strong administrative support, careful planning, a staged implementation, 
continued attention to quality control, an emphasis on customer satisfaction and, above all, a 
well-trained and dedicated staff. 

In April 2002, HSL Library Express staff celebrated a milestone. Within two years, we had processed 
more than 25,000 requests. As we gathered staff members from all three HSL libraries over pizza, 
homemade desserts and lots of stories from the “trenches,” we congratulated ourselves on a job well done 
and looked toward the continued growth of a service that emphasizes convenience and eliminates the 
barriers of distance through digital delivery to the desktop anytime, anywhere. 
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Introduction 



Most educators are innovative classroom instructors but when it comes to moving activities to the Web 
they attempt to use the exercises as they are without realizing that there are aspects of an online learning 
environment which can make those activities fioistrating for the learner and unproductive overall. Some 
online courses limit the amount of online activities, particularly those that involve peer-to-peer 
interactivity. Classroom courses that have a web component seldom incorporate online activities. 
However the power of online learning lies in its ability to enhance learning through interaction with the 
instructor and peers. 

An instructor can clearly detect when students are engaged in an effective classroom activity. Both 
energy and sound levels are higher and students are reluctant to be re-focused back to another task. The 
synergy between collaborative partners is exciting to observe, as the discussion is animated and new 
connections are established. The big question is how do you create this exhilarating learning environment 
when you “can’t see the whites of their eyes”? 



If we know what it is after the fact (when we experience or observe it), then how do we design or choose 
an effective engaged learning activity as in integral part of the curriculum? We can begin to answer that 
question by listing the characteristics that were initially defined by Dewey a century ago. He defined 
engaging activities as: 

❖ relevant to the students’ lives and experiences, 

❖ pertinent to the curriculum, 

❖ involving critical thinking and problem solving, 

❖ stimulating creative thinking, and 

❖ supporting collaborative decision making by the group. (Tanner, 1997 p. xii) 

These characteristics are still relevant to today’s classroom or online environment. When each 
component is present, the resulting holistically-designed activity brings the student and instructor together 
in an engaged, relevant, and effective task. 

The social construction of knowledge is an important element of effective activities. “The cognitive 
research reaffirms strongly that we learn best when we are actively involved in interesting and 
challenging situations, and when we talk about the learning” (Sousa, 1998, p. 8). The strategy of using 
dyads (two participants) and collaborative groups is conducive to effective learning experiences, yet the 
activity needs to be relevant to the students' interests while it challenges them to the next higher level of 
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knowledge acquisition. Discussion as a component of an effective activity assists in making meaning 
(Sousa, 1998). 



Synchronous Vs. Asynchronous Activities 

Carrie Fisher once made the observation that the problem with instant gratification is that it takes too 
long. In our fast paced world, many of us feel the pressure to have instant answers to either e-mail 
messages or the meaning of life. Many educators favor synchronous, or “real-time” interaction because 
of the influence of the classroom paradigm. Often, the instructor attempts to replicate the face-to-face 
classroom in the online environment because it is difficult to imagine how an activity works when 
learners are not participating at the same time, in the same space. 

Despite the fact that it does not happen within a shared unit of time, there are many benefits of 
asynchronous communication. It allows for reflective time prior to response. It also allows the instructor 
and learner alike to communicate at “peak” thinking periods in the day when thoughts are clearest and 
minds freshest. The individual chooses to define and interact in his or her “teachable moment. 
Asynchronous activities also allow each voice to be heard, whether in small or large group. This in and 
of itself can assist a learner in feeling part of a learning community. One danger of asynchronous 
communication is its inherent nature to go “on and on.” There must be a distinct beginning and end to 
asynchronous activities in order to minimize learner fiiistration and information overload. When used 
appropriately, the opportunity for reflective thought and communication can produce far greater depth of 
learning than would occur in a solely synchronous environment. 

There are some situations where synchronous activities are the most effective tools such as for team 
meetings or when obtaining group consensus. “It (synchronous communication) can be a dynamic and 
challenging setting in which to meet and can be especially useful in facilitating brainstorming and 
whiteboard sessions. {Whiteboarding is writing or drawing on a shared screen)” (Palloff & Pratt, 1999, p. 
48). 

Synchronous activities also can be utilized to effectively establish and maintain the learning community. 
For the first time user, the initial feelings of isolation can be lessened when the user has an opportunity to 
interact with other students and the instructor in real-time. “Synchronous communication allows the 
instructor to replicate much of a face-to-face setting with special attention applied to the need for creating 
an interactive learning community between and among all students and the instructor” (Herring & 
Smaldino, 1998, p2). 

There are many challenges associated with conducting synchronous activities. “The challenge of 
conducting a synchronous meeting or seminar is to coordinate time with a dispersed group and to 
facilitate in such a way that all "voices" are heard” (Palloff & Pratt, 1999, p. 47). In our busy schedules, 
trying to coordinate a shared time is often difficult. This challenge is resolved by scheduling established 
times for interaction. A protocol can also be established and agreed upon by all participants as to what 
part of a session is curriculum relevant and what portion is social in nature. Each session should have an 
assigned facilitator whose duties include keeping the participants on task. 

Another consideration is the structure of the discussion. It is easy for several topics to be active at the 
same time, which causes discussions to become fragmented. There also is the tendency for the fastest 
typist to be the loudest voice. The facilitator must ensure that all voices are heard. One strategy is for the 
facilitator to limit the discussion to one topic at a time and ensure that all participants have posted 
comments prior to a second comment being displayed. Numbering the participants allows easier tracking 
of this approach. A final summarization of the topic in discussion is beneficial prior to progressing to the 
next topic of interest. 
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One of the major frustrations for learners involved in synchronous activity occurs when the technology 
fails. If learners are unable to log in to a scheduled chat session they feel they have “missed class” and 
the sole opportunity to participate through no fault of their own. Logs of the synchronous discussions are 
imperative to minimize this de-motivating aspect of online “same time” communication. 

Overall, it is important to use synchronous communication only when absolutely necessary. Learners do 
not need to watch the instructor type in his or her lecture notes. Synchronous communication is needed 
for quick exchanges of thought, which is not the predominant type of thought in most learning situations. 

A combined approach may be more beneficial. For example, lecture notes could be posted for all to view 
asynchronously. Small groups could choose to interact either within a synchronous chat or asynchronous 
discussion board and post a summary of their findings in an asynchronous discussion board weekly for 
instructor comments. 

The choice of communications tool should be dependent upon the nature of the activity. Generally, 
limiting synchronous activities to a small group of participants is effective, while large group interactions 
work best within an asynchronous environment. Most importantly, the effectiveness of either choice will 
ultimately be dependent on the combination of the correct choice of the communications methodology 
and the expertise of the facilitator. 

Using Activities Designed for the Classroom in the Online Environment 

There is a need to recognize that converting a successful classroom activity into an online one is not an 
automatic one-step process. Whether an individual is riding a horse or steering a Ferrari, the road and the 
destination may be the same, but the expectations and experience will differ greatly. One approach is to 
consider each journey as a separate experience. “Indeed, instead of looking for ways to convert a course 
that has been successful in the face-to-face classroom, instructors are better served by approaching a 
course to be taught online as if it were a course to be taught for the first time-which in essence it is-while 
drawing on content knowledge and best practices” (Palloff & Pratt, 200 1 , p. 67). 

As practitioners, the authors used to believe that course re-invention was the correct path to choose. We 
now believe that many of us have thrown out too many babies with the bathwater with this philosophy. 

We need not begin from scratch; objectives are the same and therefore, the types of activities which will 
support the learning are often the same. It’s the execution of those activities that must change. For 
example, if a role-play activity is used in the classroom, the only component that changes online is that it 
becomes text-based and therefore, the rules for text-based communication must be used while the rest of 
the activity can remain the same. 

There are some common situations that can be easily converted from face-to-face into online interactions. 
Brainstorming is one of the most obvious. To quickly “shout out” phases in an online situation, use the 
subject line of an e-mail or threaded discussion to enter the “thought.” This allows everyone to quickly 
read the idea without having to open the document. Small group interactions can be facilitated in an 
assigned discussion or chat area with the instructor going from group to group and interjecting comments 
as is done in a face-to-face classroom or group reporting can be assigned to one individual for subsequent 
synchronous class discussion or summarization in an asynchronous discussion area. 

Introductory activities can be converted to the online environment by asking each participant to share one 
digital image of him or herself in an asynchronous profile or discussion area. This sharing of images can 
assist in diminishing the sense of isolation for many of the participants. Dealing with the social aspects of 
a synchronous classroom can be addressed by assigning a pre- and/or post-time slot for greetings and 
personal interactions. Allowing participants to enter a chat area 1 5 minutes prior to the actual discussion 
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elevates the existence of sign-on greetings and inquiries. Students and the instructor may also feel the 
need to process a discussion and explore new areas of thought synchronously after the assigned chat time. 
Post-discussion opportunities would allow this to take place. 

Summary 

In order to move activities between the classroom and the online learning environment the instructor must 
consider the characteristics of an effective engaged learning activity and the appropriate deliveiy 
methodology, synchronous or asynchronous. Awareness of the students’ various learning styles and 
needs also influence the choice of appropriate activities for the online environment. 

When incorporating online activities into a web-enhanced classroom the initial question asked by the 
instructor prior to conversion is whether an activity is best suited for participant interaction (synchronous) 
or for an outside class-time assignment (asynchronous). Activity selection can evolve into an exciting 
component to creating a meaningful student centered learning environment. 
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International organizations, including United Nations agencies, non-governmental organizations (NGOs), 
government donors, and university international extension programs, that are involved in humanitarian 
endeavors such as disaster relief, human rights work and international development have staff and 
stakeholder groups around the globe. Training and education of these groups are made difficult due to 
prohibitive costs, limited training budgets, travel, primary work demands that leave little for training, and 
geographical distances that separate training and education officers from their audiences. Spectacular 
advances in information and communications technologies coupled with decreasing costs can help 
overcome many of these constraints and help reach a greater number and more diverse group of people. 

International organizations now enjoy unprecedented options — such as, video conferencing, the Internet, 
and e-leaming courseware — for using technology to train, educate, inform and mobilize staff and other 
stakeholders. The Internet alone holds great potential due to the exponential increase in the number of 
users and its flexibility and growing popularity as an information and communication tool. As of 
February 2002, Nua Internet Surveys calculated that the world total population of online users was 544.2 
million people — ^an increase of over 394 million people from the 150 million online users estimated in the 
1 999 UIWP Human Development Report. Despite this fantastic increase in the number of online users in 
just three years, we are still only a little more than halfway to the billion online users projected by the 
year 2005 (UNDP 2001). 

While these innovations and trends bode well for distance learning, they also create new challenges and 
barriers to access, which can be referred to as the Internet gap, or, more commonly and broadly, as the 
“digital divide” (International Telecommunications Union 2002). Distance educators in international 
organizations must ask, “What is the nature of the divide and how will it impact the design of our learning 
programs?” “Are we building an all inclusive, widely accessible, and culturally sensitive distance 
learning program?” “How do we build a program that localizes content and builds on preferred cultural 
or personal learning styles?” “Which of our audiences will have access to this program, and which will 
not?” “How can we increase access to our program?” This paper describes the nature of the “digital 
divide” and presents creative and resourceful options that international organizations are using to bridge 
it. It is a work in progress based on our review of the existing literature and our training and distance 
learning experience. 



Disparity in Internet Access 

One of the major disparities with regard to the Internet is access, meaning our audiences’ ability to 
coimect to and access information on the Internet. At one level, this disparity exists between countries at 
different levels of development, as illustrated in Table 1. 



The Nature of the Divide 



Dufresne & Bethke ❖ 119 



ERIC 




Table 1. Disparities in Internet Access. 



World region 


Number of 
online users^ 
(millions) 


% of world 
total of online 
users 


Total 

population^ 

(millions) 


Online users as 
a % of regional 
population 


United States 
& Canada^ 


181.23 


33.3% 


320 


56.6% 


Europe 


171.35 


31.5% 


725 


23.6% 


Asia / Pacific 


157.49 


28.9% 


3,790 


4.2% 


Latin America 


25.33 


4.7% 


534 


4.7% 


Middle East 


4.65 


0.9% 


196 


2.4% 


Africa 


4.15 


0.8% 


830 


0.5% 


World Total 


544.2 


100.0% 


6,200 


8.5% 


‘ The number ol 
of February 200 
^ Estimates fron 


’ on-line users was obtained from ' 
2. 

11 the UN Population Fund (UNFP, 


Mua Internet Survey’s research as 
A 2002). 



Table 1 further illustrates that these disparities exist not only between regions, but also within them. 
Within countries, disparities are based on socioeconomic status, gender, geographic location, level of 
education and age. The 200 1 Human Development Report characterized the typical Internet user 
worldwide as: male, under 35 years old, an urban dweller, better educated, and from a higher income 
group (UNDP 2001). The digital divide, therefore, parallels socioeconomic disparities found in other 
areas of society such as education, health services, economic opportunity and welfare. 

Quality of Internet Access 

The digital divide is not just about Internet access, but also about the quality of access. Internet 
bandwidth is a good indicator of users’ satisfaction with their Internet experiences — ^that is, the greater the 
bandwidth, the quicker the response times, the more satisfied the user (International Telecommunications 
Union 2002). While high-bandwidth lines are available in many parts of wealthier nations, such as 
United States and South Korea, this technology is not widely available in many countries, especially not 
in the least developed countries. A single, sophisticated, graphic intense web page that takes seconds to 
load on a high-bandwidth line in the U.S. can take five to ten minutes in Tanzania — if the user is not 
disconnected from the Internet during the download. Distance learners limited to low-bandwidth Internet 
access will either not bother with an e-leaming course designed for high-bandwidth transmission, or will 
grow very irritated in the process of trying to take the course. 

Cost and Availability of Computers 

Over half of the world’s computers are in the United States, even though the U.S. only accounts for about 
5% of the world's population. While the cost of new and more powerful computers continues to decrease, 
computers are still a prohibitive expense for most of the world's population. A new computer with 
printer, Internet access, and various software programs costs the average income earner in the United 
States the equivalent of roughly one month’s worth of wages. The same system would cost the average 



* U.S. online users = 164.14 million out of total population of 288.5 million; Canada online users = 16.99 million 
users out of total population of 3 1 .2 million 
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Bangladeshi eight years’ of income (UNDP 1999). Most people in the world will never be able to afford 
their own computers, and most countries will not be ale to invest in the infrastructure or training required 
for efficient Internet usage (Uimonen 1997). E-leaming courses and programs that implicitly assume 
unlimited access — both to personal computers and the Internet — ^will fail to reach audiences who must 
rely on expensive, sporadic Internet connections and group computers. 

Language and Communication Barriers 

Access and quality are both technological and economic challenges. Language and cultural differences 
present a different kind of challenge for the distance educator. One study estimates that approximately 
40% of online users speak English while the remaining 60% speak some thirty other languages. After 
English, the next three largest online language populations are Chinese (9.8%), Japanese (9.2%), and 
Spanish (7.2%) (Global Reach 2002). In 1999, one study estimated that 80% of Websites were in English 
(UNDP 1999). Since people learn and express themselves best in their native language(s), e-leaming 
courses delivered solely in English and meant for global audiences may fail to achieve their hoped-for 
impact. 

Language is just one communication barrier. In addition, the Internet experience reflects a particular 
worldview and style of communication — ^that is, detached, individualistic, text/graphic based, technology- 
dependent, specific, democratic, decentralized and functional. The computer and Internet based e- 
leaming experience also depends heavily on a confident, computer literate learner who values democratic, 
individualized and self-directed teaching and learning experiences. For those who are accustomed to, or 
who value, traditional, expert-led or professor-led classroom teaching, this kind of an experience can 
seem anarchic, superficial and/or threatening. E-leaming has been slower to catch on in places (Middle 
East, South America and Southern Europe) where the culture and learners favor emotionally charged 
debates, holistic ‘ big picture’ learning approaches, live experts and group collaboration (Quigley 2002). 

Bridging the Divide 

The above are just a few of the challenges that face teachers, trainers and course designers who are 
creating e-leaming courses for global audiences. Challenges, like necessity, beget creative and 
resourceful solutions, which are the focus of this section. 

Use and Establish Public Information and Communication Centers 

Thousands, if not millions, of people in many developed and developing countries have access to 
computers and the Internet but only via their workplace computers, Internet Cafes, or community 
telecommunication and information centers. E-leaming programs expecting to reach audiences in these 
countries must recognize this opportunity and build on it. 

For example, the MS Swaminathan Research Foundation developed an innovative project in Pondicherry, 
India that allows fishermen and farmers to obtain useful market and other business-related information at 
mral information centers. These centers use solar and electric power and provide local communication 
and Internet access via wired and wireless communications. They have facilitated communication 
between villages and “have encouraged local dialogue on farming techniques, microcredit management, 
business and education opportunities, traditional medicine and religious events. About one-third of users 
are from asset-less households and about 18% are women” (UNDP 2001 : 32-33). Community 
information and communication centers are just one example of an innovation that bridges the Internet 
Divide and that should be, and are being, duplicated by NGOs, churches, foundations, donors and 
governments. 
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Create Flexible Distance Learning Options 

Gcxxl instructional design does not begin with the technology, but rather with an analysis of the course 
purpose, objectives, audience, budget, technology options, cost, etc. A flexible delivery strategy, using 
different media, including the Internet, can be especially effective when one wants to accommodate 
diverse audience needs and preferences, and to anticipate near-term trends. Brigham Young University’s 
Teaching English as a Second Language (TESL) Distance Learning Project used a flexible and combined 
“high and low road” technology approach to meet the learning needs of a diverse group of volunteer 
English tutors (Henrichsen and Murray 1995). Most of the tutors owned a VCR and considered video 
their instructional media of choice. Therefore, a fundamental component of the project was a videotape 
series featuring a variety of teacher preparation topics presented using situation dramatizations followed 
by reflection. There was also a companion workbook that could be used in conjunction with or 
independent of the video. Because the project developers recognized that, in the future, more and more of 
their audience would be using computers and the Internet, they also planned to use more sophisticated 
media including CDs, interactive computer applications and the World Wide Web using 
hypermedia/hypertext. 

Rely on Low-Bandwidth Internet Options 

Even in places where high-bandwidth is not available and where connectivity charges are high, low- 
bandwidth Internet design options do exist. Such options include: 1 .) Web pages with minimal and small 
file size (or optional) graphics, no (or optional) video and audio streaming, no frames and a user-friendly 
interface; 2.) Web-accessed and text-based (versus PDF) course files, readers, documents, and worksheets 
that can be downloaded quickly and then viewed, read and completed off-line; 3.) Learning communities, 
learning experiences, teacher/leamer communication and information sharing that can be managed and 
accessed simply by using e-mail, as opposed to connecting to a Web page, since in poorer countries e- 
mail extends further than any other Internet-based technology. 

In a similar vein, many distance learning programs have been implemented successfully, over the years, 
using low technology options that in many areas are more widely available than the Internet, such as 
print-based modules, cassette tapes, radio, television, and videos. In many cases, even where audiences 
have Internet access, their schedules, work and learning style may still favor these lower-tech options. 

The authors, for example, were involved in a WebCT self-study program that was based on print-based, 
independent study modules. While all participants had easy access to the WebCT course, an evaluation of 
this program revealed that the majority of participants who enrolled and completed the course did so 
using, not the WebCT option, but the print-based module which they also received when they signed up 
for the Internet course. 

Use a Blended Approach 

Blended learning approaches that combine traditional classroom with online and/or workplace learning 
experiences can serve to reinforce classroom learning, reduce classroom time and costs, and 
accommodate different learning needs and values (e.g. self-directed, group activity, teacher-led, social 
needs, pragmatist). A recent pilot blended training-of-trainers learning program designed for the 
International Labour Organisation by one of the authors consisted of a face-to-face workshop, preceded 
and followed by distance learning components managed entirely by e-mail and file attachments. 
Participants in this program were stationed in Lima, Peru; Dar es Salaam, Tanzania; and Geneva, 
Switzerland. Prior to the workshop, program participants received, via e-mail, a distance learning packet 
with instructions, a reader, assignment worksheets and deadlines for submitting completed assignments. 
After the workshop, program participants received another distance learning packet, again via e-mail, 
with assignment worksheets and guidelines instructing them to design and evaluate an actual training 
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workshop or major training presentation that they were delivering in the upcoming four months. Upon 
completion of this training experience, participants submitted their completed worksheets, personal and 
peer evaluations, and report of lessons learned from their participation in the program. 

Develop Local Facilitators 

To overcome problems of language, cultural learning preferences, and costs associated with Internet- 
based global distance learning, course designers can recruit, train and prepare local facilitators. These 
local facilitators would be able to download training materials from the Internet course web site that they 
can then modify for local audiences. The Internet thus serves primarily as a distance training and 
resource tool for local facilitators who, in turn, train, educate and mobilize local audiences using print- 
based self-study and face-to-face workshops. In 1997 the World Bank Institute launched the World Links 
for Development, or WorLD, program, which has since been established as a separate nonprofit 
organization. The WorLD program aims to establish global, educational on-line communities for 
secondary school students and teachers around the world to expand distance learning opportunities, 
enhance cultural understanding across nations, build broad support for economic and social development, 
and train teachers to integrate information technology into the classroom. A key to the development of 
new pedagogical methods for teachers relates to the system of teacher training used by WorLD. WorLD 
Project Staff, who have easier access to the Internet and other high tech solutions such as video 
conferences with experts from other countries, train local teachers and school-based WorLD Coordinators 
in new pedagogical methods and in how to use technology to enhance their teaching. In addition, informal 
training among teachers is helping to bring about a change in pedagogical practices (McGhee 2001). 

Conclusions 

Many e-leaming courses make erroneous assumptions about computer and online users and the quality 
and ease of their access to the Internet. E-leaming programs aspiring to reach diverse global audiences 
will need to address existing economic and technological disparities as well as the diverse learning styles 
of users. Distance educators need to know their audiences and create ©-learning experiences that are 
accessible; user friendly; interactive; locally relevant and sensitive to language, communication and 
learner preferences. Creativity and resourcefulness in instructional design can address technological 
constraints and socioeconomic disparities to harness at least some of the promise and power of 
technological innovations. 
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To create a statewide system so that all Ohio students, regardless of school, geographic location 
and socioeconomic background, have access to high-quality, concept-driven learning 
experiences. This state-wide system is intended to support and extend the overall state ’s vision of 
systemic reform in education. 



The Ohio SchoolNet Telecommunity Vision 



Overview 

Historically, the Ohio SchoolNet Telecommunity (OSN TC) program has worked toward that vision by 
providing grants to help schools implement interactive video distance-learning technologies that link 
students with other schools, institutions of higher education, fine arts institutions, libraries, zoos, and 
other educational service providers. The OSN TC also provides grants to non-public and parochial 
schools to support the implementation of Internet technologies and information resources into classroom 
teaching and learning. 

The OSN TC was originally founded in 1995 as one result of an “alternative regulation” proceeding and 
agreement between the Ohio Department of Education (ODE), the Public Utilities Commission of Ohio 
(PUCO), and Ameritech Ohio. In 1996, eight additional telecommunications providers joined the Ohio 
SchoolNet Telecommunity, also as a result of an agreement between the companies, the PUCO and the 
Ohio SchoolNet Commission. Participating telephone companies include: ALLTEL/Westem Reserve, 
Ameritech/SBC, CENTURYTel, Chillicothe Telephone Company, Cincinnati Bell Telephone Company, 
Verizon, Orwell Telephone Company, and Sprint/United. 

Oversight of the OSN TC is provided by a 21 -member Policy and Oversight Committee consisting of 
diverse stakeholders including representatives from state university education faculty, Ohio educational 
technology agencies, Ohio educational television, the Catholic Conference of Ohio, ODE, the PUCO, the 
Ohio SchoolNet Commission, Ohio K-12 schools, telephone company funding providers, and other state 
and national experts. The Policy and Oversight Committee provides vision setting, oversight of grants, 
policy and planning, program evaluation, and approval and release of all program-related materials. Day 
to day activities are coordinated by a Program Manager and support staff at the Ohio SchoolNet 
Commission. 

Through the first 5 years of the program, the OSN TC has provided more than $27 million in funding to 
approximately 475 schools in over 35 consortia for interactive video distance learning projects in 
furtherance of the OSN TC vision. The OSN TC also supports non-school content providers such as 
museums, zoos and other entities that provide K-12 educational resources. The Program has awarded 10 
content development grants totaling approximately $3 million. In addition, 160 grants of up to $8,000 
each have been awarded to non-public schools and community schools for the implementation of Internet 
projects. The program is expected to distribute over the life of the program more than $32 million to 
Ohio schools. 
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The Ohio legislature, with Ohio SchooINet as the implementing agency, has provided over $850 million 
in additional infrastructure and educational technology programs. SchooINet Plus+ has provided 
computers through grades K-6; the ONEnet infrastructure and connectivity program has provided ATM 
switching and T-1 level bandwidth support; and the Interactive Video Distance Learning program has 
resulted in an additional 350 schools provisioned with video distance learning equipment. Although the 
OSN TC vision remains that of promoting the establishment of a statewide interconnectable distance 
learning network, future programmatic support, will also include the nurturing of existing projects and 
schools with video distance learning equipment, as well as maintain a focus on high-quality content and 
professional development for existing program participants. 

The Program 

In the early days of the Telecommunity there were virtually no models of interactive video connectivity 
being used in the K-12 classroom with the exception of a small pilot project in northeast Ohio sponsored 
by Ameritech Ohio. That pilot focused on using video technology to expand high school course 
offerings, especially in foreign language instruction, where several schools might have a few students 
interested in a certain course that could not be offered because there were not qualified teaching staff 
available in each school. The OSN TC was established as a somewhat experimental competitive grant- 
giving program to expand the concept of teaching and learning via video conferencing technologies 
beyond the offering of specialty courses. The original planning and implementation grant opportunities 
requested proposals for new models and uses of video technology. 

The Policy and Oversight Committee was and is interested in models that not only recognize specific 
course instruction, but also professional development, teacher collaboration, community involvement, 
career development and unique “field trip” opportunities to access educational resources and information 
otherwise impossible to access. The first grant opportunities included a “planning grant” and an 
“implementation/content grant.” The planning grant provided support for a consortium of schools and/or 
content providers to research, design and submit an implementation project proposal. The 
implementation/content grant opportunity requested proposals from consortia of schools and content 
providers that demonstrated innovative and “cutting edge” models of using interactive video distance 
learning in the K-12 classroom and in the teacher professional development process. Support was 
provided for video and affiliated equipment, bandwidth infrastructure, oversight and management, 
professional development and content acquisition and development. More recently grant opportunities 
have been expanded to both new consortium applications and support for existing projects. 

The original agreements with the PUCO and the ODE stipulated that funding would also be provided for 
non-public and parochial schools. While funding was provided for one large video-based experiment 
involving Ohio parochial schools, the main thrust of support for private and parochial schools is through 
the “Internet grant” funding opportunity. Non-public and parochial schools submit proposals for 
classroom teaching and learning projects that involve the use of computers in student activities and the 
use of Internet resources and information in the learning process. 

At present the OSN TC offers the following funding opportunities: 

❖ Implementation Grant RFP (public school consortia only) 

❖ Content Grant RFP (non-K12 school content providers only) 

❖ Support for Existing Projects (existing implementation and content projects) 

❖ Adding Partners to Existing Implementation Grant Projects (implementation grants only) 

❖ Internet Grants (non-public and parochial schools only) 

❖ Professional Development Grants (non-public and parochial schools only) 
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Grant opportunities are offered twice each year, with RFPs and application materials being released in 
November and May. The Policy and Oversight Committee votes to release appropriate funding 
opportunities at every regular meeting. Not all RFPs are offered each grant round. Application deadlines 
are generally mid-March and mid-September. Letters of Intent are required for most grant opportunities. 
Contracted agreements are required of all consortium participants, outlining the requirements and 
deliverables for each award. Implementation and content grants are generally awarded as three-year 
projects, while Internet grants, professional development grants, support for existing project and adding 
partners are under one-year implementation timeframes. 

The Evaluation Process: Criteria for Success 

The importance of program evaluation is such that the OSN TC original agreements included an annual 
independent evaluation component. The annual evaluations look to review not only the operations of the 
program but also the effectiveness of grant implementation and the impact of interactive video 
technologies on the teaching and learning process. The first five annual evaluations were conducted by 
the North Central Regional Educational Laboratory (NCREL.) The Year 6 and 7 Evaluations will be 
conducted by Metiri Group. A report document is provided to the Telecommunity Policy and Oversight 
Committee and the Ohio SchooINet Commissioners each year and is posted to the OSN Website. 

The most recent evaluation looks at measures of successful interactive video Telecommunities (TCs) and 
reports several criteria for success: 

1 . Vision and Mission 

❖ Successful TCs have a strong, clear vision, widely shared and clearly communicated, for how 
IV technologies could be used to advance student learning. 

❖ They have clear vision and mission-important determinants of the degree of school and district 
ownership, buy-in, and administrative responsiveness, as well as the extent to which the TC is 
embraced by teachers. 

2. Staffing and Organizational Structure 

❖ Successful projects have full-time managers/staff dedicated to the TC. 

❖ They establish Committee structures clearly aligned to the overall mission and vision of the TC. 

❖ They include a wide range of administrators and teachers in the TC governance structure. 

3. Instructional Support 

❖ Successful projects have a support system that provides general technical and instructional 
services, including Just-in-time classroom-based technical assistance and pedagogical support. 

❖ Successful TCs have systems of teaming, teacher partnerships, and PD that ensure adequate 
training and support for classroom strategies required for participation in the TC. 

4. Professional Development and Training 

❖ Successful projects have strong, collaborative professional development programs that use 
interactive video as one method of delivery among others. 

❖ Successful TCs use professional development to facilitate inter-site collaboration and to instill a 
sense of ownership in each participating teacher. 

5. Funding and Outside Resources 

❖ Successful projects have the ability to leverage additional funds and resources from a variety of 
sources. 

❖ They are savvy about bringing in partners and programming that add value to both the mission 
and participating schools. 
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Successful TCs aggressively seek additional funds to support their goals. 



6. Programming and Content 

❖ Successful projects include an abundance of quality interactive video-based content drawn from 
a variety of sources, including intra-consortium partnerships, strong outside content providers, 
interactive partnerships outside the consortia, and university/consortia partnerships. 

7. Changes in Teaching and Learning 

❖ Successful projects generally show evidence of changes in teaching and learning attributable to 
the Telecommunity. 

* 1 * Successful TCs consistently demonstrate that more student-focused missions are more likely to 
produce stronger results. 



The Ohio SchoolNet Telecommunity (OSN TC) is a unique public-private partnership funded by 
SBC/Ameritech and Ohio’s other large local exchange telephone companies. The program has 
historically and continues to provide grants and other support to assist Ohio schools implement interactive 
video distance-learning technologies that link students with other schools, institutions of higher 
education, fine arts institutions, libraries, zoos, and other educational service providers. While it is fair to 
say that interactive video technologies are still somewhat experimental and are by no means completely 
accepted and integrated into the Ohio K-12 curricula and teaching and learning process, the OSN 
Telecommunity program has demonstrated the tremendous value of video technologies to schools and 
will continue to make effective use of existing resources, both within the operational TC program 
agreements and through any resources that might come available, to support and promote interactive 
video distance learning as a tool to extend and enhance the educational experiences of Ohio students. 
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Success in distributed learning and all academic programs depends a great deal on faculty adoption and 
readiness (Waterhouse and Harris, 2002). The demands on faculty and staff who use technology in their 
courses are continuing to rise. Providing an environment which is supportive of faculty and contributes to 
their eventual success is critical. Distance education is an area where faculty members need assistance in 
learning how to use the new technologies (Cyrs, 1997; Thach & Murphy, 1995). Development of 
technology competencies can form the basis of good faculty professional development programs (Blue, 
Greer, Vetter, Irvine & Cole, 2000). While it is important that faculty understand how to use the tools of 
technology, it is equally Vital that they understand the pedagogies underlying these tools (Laurillard, 

1993; Palloflf & Pratt, 1999). 

The Division of Instructional Services at Collegis has worked with more than 70 clients in the past four 
years to develop faculty training and support strategies. We start with a good foundation via general 
pedagogical standards such as the Seven Principles of Teaching and Learning (Chickering & Gamson, 
1996), the regional accreditation agencies’ Principles of Good Practice (Council of Regional Accrediting 
Commissions, 2000), and general web usability standards. Faculty need to understand why they are 
learning the best practices and how these transfer to the web (Cravener, 1999). Often, they find that they 
can use what they learn in their traditional face-to-face classrooms as well (Fink, 2002). Sometimes 
institutional or program standards evolve collaboratively between the institution and the instructional 
designer. We find that faculty members are often relieved to hear about standards because it gives them a 
framework for course design so they don’t have to reinvent the wheel. 

Faculty development can be accomplished through either centralized or decentralized support (Ramsey & 
Breeden, 2001). All resources can be housed in a centralized office, and staff assigned on a first-come, 
first served basis. Or staff can be housed centrally, but specific individuals are dedicated to their own 
departments. A third alternative is that faculty development resources work in their own departments. 
Finally, if an institution has a contract with an outsourcer, the company can provide support through on- 
site staff or through centralized services, or a combination of both. 

A variety of topics and competencies are covered during the course of faculty development. We have 
seen these competencies change over time. For example, initially institutions wanted more tools training. 
Now that many have faculty that know the basics of tool functionality, a desire exists for more pedagogy 
and train the trainer, as well as mentoring. Other issues include: Is training different if blended/hybrid 
courses are taught vs. fully online? What happens when you train different instructional technologies, 
such as interactive television, or face-to-face support — ^how many of the competencies change? The tools 
may change, but does the basic pedagogy stay the same? Should faculty be required to learn it all, or 
work in teams where instructional designers facilitate the development of their courses (Fink, 2002)? 

Are there ways to facilitate training, such as training to a specific curriculum template? 

During the course of faculty development, we have discovered the value of: 
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❖ developing a training plan to inform the appropriate parties about what will be taught, and 
to whom, and by which staff during speciflc timelines (Bender & Craymer, 2001) 

❖ faculty learning from each other - opportunities to share experiences (peers) or experienced 
faculty presenting to novice faculty 

❖ showing many good examples in the discipline to give faculty a concrete understanding of 
what can be done 

❖ working with early adopters to train or positively influence other faculty members 

❖ training students to make faculty workloads lighter. 

When courses are developed for faculty, we also need to continue to apprise faculty about teaching 
strategies in general, as well as about online course design, in particular. This includes everything from 
instructional design to web design to the nitty-gritty processes for designing and developing efficiently, 
using a team approach, and adhering to a defmed timeline. Specific examples of what we teach include; 
learner-centered vs. traditional approach to teaching/leaming, how to define observable outcomes and 
write learning objectives, how to incorporate discussion assignments and write effective discussion 
questions (general guidelines as well as discipline-specific variations), how to create effective group 
assignments, how to incorporate real-world scenarios and applications into assignments, how to 
incorporate opportunities for self-assessment (embedded questions with feedback, self-tests), how to 
communicate content via images and animation, how to use audio and video effectively, how to 
incorporate e-pack or course cartridge components from the publishers, how to structure an online course 
for usability, and how to use a prototyping process for efficient development. 

Some of the lessons we have learned include: the importance of flexible scheduling (just in time coaching 
and use of a range of formal training times, from two-hour workshops to five day faculty “bootcamps “). 
Schedule early enough for faculty to work on their projects, but not too early that they forget what they 
have learned. Use the technologies involved to train faculty. One of the most critical things we have 
learned is to group cohorts with similar pre-requisite skills whenever possible. Also exposure to 
technologies should be limited for mainstream faculty who have not been on the leading edge of 
technology use — e.g. they shouldn’t have to learn HTML. And provide faculty with just-in-time online 
and people resources they can refer to whenever they have a problem. 

We have encountered the usual barriers to development: compensation and/or release time, time for 
training itself, lack of motivation for getting training, recognition for promotion and tenure, ownership, as 
well as infrastructure. It is important that the goals for development are effectively articulated. Faculty 
need to be involved in the planning process for adopting new technologies and using them effectively, 
while understanding the value of what they are doing. Communication is key! And if a teaching and 
learning center exists at the institution which does not focus or involve technology training, work closely 
with its staff to bring technology into the equation. 

It’s important to remember that not one set of strategies works for every faculty development situation. In 
order to be successful, however, faculty must be informed of how they can benefit from instructional 
technology and distance learning, and receive timely support where and when they need it. 

References 

Bender, C.L, & Craymer, S. (2001). Starting over from the middle: Constructing a faculty development 
plan. Paper presented at the 17* Annual Conference on Distance Teaching & Learning, Madison, 
WI. 




130 ❖ Faculty Development Models 



132 



Blue, G. H., Greer, E., Vetter, R. J., Irvine, T, & Cole, B. (2000). Final Report for the Distance Learning 
Design/Model (DLD/M) Project. Paper presented at Instructional Technology Council 
Teleleaming conference, Atlantic City. NJ. [On-line]. Available: 
http://people.uncw.edu/vetterr/ncccs/ncccs.doc 

Chickering, A. W., & Ehrmann, S. C. (1996). Implementing the seven principles: Technology as lever. 
[On-line]. Available: http://www.tltgroup.org/programs/seven.html 

Chizmar, J. F., & Williams, D. B. (2001). What do faculty want? Educanse Quarterly, 1 , 18-24. [On- 
line]. Available: http://www.educause.edu/ir/library/pdf^eqm01 12.pdf. 

Council of Regional Accrediting Commissions (2000). Statement of Commitment by the Regional 

Accrediting Commissions for the Evaluation of Electronically Offered Degree and Certificate 
Programs [On-line], Available:. http://www.wiche.edu/telecom/Accrediting%20- 
%2 OCommitment.pdf 

Cravener, P. (1999). Piloting the psychosocial model of faculty development. The technology source. 
[On-line]. Available: http://ts.mivu.org/default.asp?show=article&id=563 

Cyrs, T. E. (Fall, 1 997) Competence in teaching at a distance. New Directions for Teaching and 
Learning, 71, 15-18. 

Epper, R. M,.& Bates, A.W. (2001). Teaching faculty how to use technology: Best practices from 
leading institutions. Westport, CT: Oryx. 

Fink M. L. (Feb., 2002). Rethinking faculty support services. Syllabus, [On-line]. Available: 
http://www.syllabus.com/syllabusmagazine/article.asp?id=6076 

Gillespie, K. H., Hilsen, L.R., & Wadsworth, E. C. (2002). Guide to faculty development: Practical 
advice, examples, and resources. Bolton, MA: Anker. 

Laurillard, D. (1993). Rethinking university teaching: A framework for the effective use of educational 
technology. London: Routledge. 

Palloff, R. M., & Pratt, K. (1999). Building learning communities in cyberspace: Effective strategies for 
the online classroom. San Francisco: Jossey-Bass. 

Ramsey, E., & Breeden, S. E. (2001). One size does not fit all: Two models of support and training. 

Educause Quarterly. [On-line]. Available: http://www.educause.edu/ir/library/pdf^eqmO 132.pdf 

Thach, E. C., & Murphy, K. L. (1995). Competencies for distance education professionals. Educational 
Technology Research & Development, 43,{\), SI -19. 

Waterhouse, S., & Harris, R. (2002). A ten-step guide to establishing instructional technology. 
Washington, DC: Executive Leadership Foundation. 

Biographical Sketch 

Edrie Greer is Director for the Instructional Services division of Collegis, where she provides leadership 

and management for the faculty development services for the company. She is also a visiting lecturer for 

the East Carolina University School of Education's Librarianship, Educational Technology and Distance 



133 



Greer 



131 



Instruction department, where she has developed and taught courses and workshops online and via 2-way 
interactive TV (NCIH) since 1995. Previously, Edrie coordinated distance learning activities for the 
Division of Continuing Studies at East Carolina University, and was a Distance Learning Specialist and 
Producer- Writer for the University of Wisconsin-Extension and Wisconsin Public Television and Radio. 
Edrie holds a bachelor’s degree in Natural Science Journalism and a master’s degree in Continuing and 
Adult Education from the University of Wisconsin-Madison, as well as a doctorate in Philosophy from 
the New Thought Theological Seminary. 



Address: 


2000 Perimeter Park, Suite 160 
Morrisville, NC 27614 


E-mail: 


egreer@collegis.com 


URL: 


www.collegis.com 


Phone: 


919.376.3431 


Fax: 


919.462.6450 



ERIC 



132 ❖ Faculty Development Models 



131 



Moving From Online Courses to Online Programs 
Is Not for the Faint-Hearted 

Lois S. Hale 

Chair, Department of Behavioral Science 
The University of Texas of the Permian Basin 

Douglas F. Hale 

Director of the REACH Program Center 
The University of Texas of the Permian Basin 

Introduction 

What happens when an institution moves from offering a somewhat random collection of online courses 
to delivering entire degree programs exclusively online? What are the rewards and the challenges? How 
can these challenges be met? These are the frmdamental issues that will be discussed in this paper. 

The primary objectives of the paper are outlined below along with a couple examples of relevant issues. 

1. To outline some of the student support issues that institutions face when offering a degree program 
entirely online - academic advising, remote library access, acquisition of course materials. 

2. To suggest proactive strategies that may ease the transition from on campus to online degree 
delivery — campus-wide advisory group, single faculty or other point of contact for online students, 
proactive communication with the online students. 

3. To suggest reactive strategies when the courses are ready, but your institution is not prepared to offer 
all of its services to the online students - priority handling of online students, cultivating institutional 
advocates, developing an “I’d kill for their email addresses” mentality. Interestingly enough, some of 
the more iimovative strategies come from the need to react to an immediate need. 

4. To discuss the politics of institutional change - why you shouldn’t wait until everything is perfect 
and all systems are “go” to launch the online program, what unexpected benefits might arise to 
traditional students, and how to keep the needs of the online students the center of your focus. 

Institutions are encouraging faculty participation in online instruction by suggesting that they begin by 
enhancing their on campus courses by adding some Internet technologies (e.g., lecture notes online, 
asynchronous discussions, Web and data-base searches). As faculty become more comfortable and 
competent, they may be encouraged to move their courses to a delivery that is totally online. Typically, 
however, the students in these courses are dormitory or commuter students. For them, the availability of 
Web-enhanced or Web-based courses serves to decrease scheduling conflicts, increase convenience, and 
reduce commuting time. 

What happens when the institution moves from offering a somewhat random collection of online courses 
to offering a full degree program delivered exclusively online? That is the subject of this paper, and the 
quick answer is “a whole lot.” 

Before looking at the specific topics, here are some general difficulties you will face. These are also 
reflected in the specific examples given later in the discussion. First and foremost, and probably already 
known to you, is the fact that your institution is not yet used to thinking institutionally in terms of distant 
students. A second and related difficulty is that the external organizations (for instance, governing boards, 
accrediting entities, state curriculum oversight groups, legislators) that impact program development are 
not any more advanced in their understanding of distant students and programs than your internal 
organization is. Some do have supportive stances. Governing boards and legislators like the concept 
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because it opens access. At least governing boards of state institutions and their legislative friends like 
access to programs in their state by their citizens. In Texas, we do not seem to care much about attracting 
students from anywhere other than Texas. That’s why we continue to charge non-Texas resident tuition 
and fees to distant students, but that’s another story. 

The remainder of this paper is organized into sections: enrollment, authentication, communication, and 
marketing. Within each section, the issues as they relate to the development and delivery of an online 
degree program are discussed, reactive strategies are shared, proactive solutions are suggested, and tactics 
for winning the game of institutional politics and policy are recommraded. 

Enrollment 

So you’ve decided to put your program online. Your potential students will not be coming to campus for 
other courses. They won’t drop by to pick up a fall schedule on their way from the dorm to the library. 

Are you set up to handle admissions, academic advising, registration, billing and financial aid at a 
distance? We weren’t. One of the first things you realize is how many forms and procedures are paper- 
based and how rigid an institutional mentality that suggests the student “drop by” and fill out the form can 
be. But there are signs of progress. The forms, many of them at least, are now available online, usually in 
some handy format like pdf, so you can print them off, fill them out, sign them, and drop them by the 
office at your convenience! Right! 

When our first program went online, the institution had no phone or online registration. The few distance 
students we did have faxed in their registration cards. Our reactive response as program administrators 
was to develop our own electronic versions of the admission and registration forms and to put them on 
our program Website. We then offered the electronic versions of the forms to the various units for their 
use. Not only did we provide an electronic version, we added a field for email address to every form as 
part of the required student demographic data. Proactive strategies certainly included raising the 
awareness of the institution to the fact that all students would appreciate and use enrollment services 
offered at a distance. As part of that strategy we built alliances. For example, the computer services folks 
didn’t really want to drag computers to the gym for registration every semester. They were a bit tentative, 
but did join us in the call for institution-wide Web based registration. 

Our biggest enrollment issues continue to be in the accounting office. It may just be our institution, but 
accounting’s attitude seems to be “Don’t pay your tuition. Fine. We’ll just cancel your registration and 
drop you from your courses.” If that happens during the first few days of the semester, it means that the 
student’s access to the course is immediately denied. Do that on a Friday, and a student is shut out for the 
entire weekend. Reactive strategies included finding one sympathetic accountant, explaining over and 
over again that these students are not going to drop by to pay their bill, and providing that accounting 
person with email addresses for all online students. Our sympathetic accounting person has even floated 
students emergency loans without them knowing it, just to save their registration and course access for a 
couple of extra days. Proactive strategies include simply working with accounting to help them to think 
through what the distant students see from their end. Unfortunately, sometimes it’s hard to anticipate 
changes that may negatively affect online students. One semester under a new Vice President for Business 
Affairs, the institution decided not to mail bills. Another semester we mailed bills that had no accounting 
contact information and no due date. Changes necessary in billing procedures resulting from our 
implementation of the Web based registration system simply escaped our poor accounting folks. 

The single most successful strategy when dealing with any institutional unit that provides student support 
services to distant students seems to be the development of a relationship with a sympathetic point person 
in that unit, whether it be admissions, registration, accounting or graduate studies. Do the same thing with 
student advising. Designate one faculty or staff person to advise online students at least in the early stages 
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of their degree program. If that person has a good understanding of and rapport with the other support 
units - admissions, accounting, etc., then you will be able to provide even better student support services. 

A second strategy is to form an institutional committee made up of representatives from every unit whose 
services impact distant students. Our committee is chaired by our distance education program director 
who reports to the Vice President of Academic Affairs. The committee includes faculty representatives 
from each online program and administrative staff representatives from distance education, computer 
services, admissions, registrar’s office, financial aid office, accounting office, public relations and the 
library. You might consider adding bookstore personnel, too. Meeting two or three times a semester, this 
committee is a good place to resolve recurring problems, brainstorm solutions for potential future 
problems, and build support for the distance education enterprise and empathy for the distant student. So 
many times a change in a procedure in one unit will impact not only the distant student but also another 
unit. In a very proactive way, these potential problems can be addressed and resolved among these 
committee members. 



Authentication 

“How do I convince people that I exist and am legit?” Everybody who has ever had control of a computer 
system and was responsible for providing secure access to it apparently believes the student would only 
ever want to access that one system and so will not be overburdened by keeping track of proper 
authentication information, usually a username and password. Until very recently, a distant student on our 
campus typically would actually have a username and password for the University’s central computer 
system, the software system supporting course delivery, the on-line student information system, and the 
electronic databases provided by the library. If the student were taking a UT System TeleCampus course, 
then she would also need usemame/password combinations for the TeleCampus course delivery system, 
the student information system, and the System Digital library. One shudders to think about the student 
undertaking course work from more than one campus. Of course, the student is cautioned not to write any 
of this down anywhere, but to simply remember the information. Good question, but no, that won’t work, 
either. You shouldn’t use the same name and password all over the place anyway, and some of the 
systems insist on your using name and even passwords derived from a system-specific formula. 

Encourage the information systems powers that be to consider a one-time campus sign in procedure that 
will allow access to the instructional software, the library, the student information system, and anything 
else the student may need having to do with your institution. This can be done using a proxy server, a 
device which can be signed into once and then used to fool the other services into thinking the student is a 
valid user. Unfortunately, it is often the case that the student’s browser must be specially configured to 
deal with the proxy server. That is fine, if access is only desired to one place. If the student needs access 
to two proxy servers, the use of Netscape for one and Internet Explorer for the other is recommended. We 
have no answer for cases in which the student needs access to three proxy servers. You may also note that 
this may get some students in over their computer literacy level. Ah well, they are here to learn, aren’t 
they? We recommend you investigate something like EZProxy, a proxy server that does not require 
special browser configurations. 

A slightly different authentication issue appears when the student takes a course in which some sort of 
proprietary software is required. Since a word processor seems a nearly universal course requirement, 
almost all students are affected. In our situation, the UT System, like many other such institutions of 
higher education, has struck a deal with a major vendor to provide basic office automation software to 
students (and faculty and staff) for the price of, well, free. That is, the System has sent a large check to the 
vendor and students can purchase the software through the bookstore for the price of the CDs it comes on. 
Just drop by and get it when you drop off your registration card and please be sure to bring your student 
identification card with you so you can prove who you are. Our deal with the vendor is that you only get 
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one copy and we must know that you are actually a student. These are understandable concerns for the 
vendor and therefore for the bookstore. Our first reaction was a rather cumbersome, but workable, scheme 
involving the student sending (faxing) a copy of a driver’s license, a signed EULA, and a credit card 
number or check (Sorry, checks by fax are not accepted.) to the bookstore and the bookstore checking 
student rosters to verify enrollment. An online system allowing the student to order and pay for software 
and other course materials, with a mechanism to check the student rosters for enrollment verification 
would be better. 



Communication 

The need for communication among academic and service units at the institution as well as the need to be 
proactive in communications with students is real. Initially, nobody, in our institution had any experience 
with many of the problems discussed here and, of course, they were eager to learn - or not. Anyway, 
communication among the players (administrators, faculty and students) is crucial. As we mentioned 
previously, you should form a committee to encourage the sharing of information and include 
representatives from each of the academic and support areas involved in distance education. It is also 
important to identify someone in each office dealing directly with students as the ‘‘distance education” 
person. Dealing with distant students is very different and many procedures will have to change. 

With respect to faculty/student contact, problem number 1 is getting the student’s attention in the first 
place. Yes, you could just email them, send them enough information to get them started, and ask for their 
email address, if you had their email addresses. If you don’t, you can just tell them (at the first class 
meeting ???) to send an email to you and you will capture the address. Our reactive solution has been less 
than wonderful. Our institution provides phone numbers on class rosters, so the students can be called and 
asked to send contact information. Obviously, this process will not scale very well. We have begun to 
collect email addresses from students through the Web based registration system. We print them on the 
hard copy class rosters sent to faculty. 



Marketing 

When student recruitment means visiting the local high schools and taking out an ad in the local paper, 
there will be a need for an institutional “rethink” about marketing. Marketing efforts aimed at attracting 
students who are far from your campus and who will remain so when they “come” to your institution are 
different in both method of distribution and in content. Unless your institution is going entirely online, 
you should target learners by program. For instance, the UT TeleCampus advertises in places such as the 
Southwest Airlines magazine, with copy aimed at the business traveler who might be interested in an 
MBA. There need be no pretty pictures of campus flowers and students and buildings. The message is 
that you can do the program while you are doing vAiat you are doing right now - traveling on business. 
For the online kinesiology program, advertising is placed in professional journals and distributed at 
regional and national conventions of kinesiology professionals. This is a good model for any graduate 
level program and may be for some undergraduate programs as well. 

However they are done and wherever they appear, real advertisements are expensive, so alternatives are 
important. Question: where do you think the prospective online student in likely to look for information 
about programs at a distance? Yes, of course, so it is important for you to set up an attractive Website for 
your program and to keep it current. One thing you want to be sure to do is to include descriptive 
information in your page metatags so the average search engine will easily find your site. Get your local 
Web administrator to place a link on the institution’s front page labeled something like Distance Learning 
and leading to a page on which your program will be obviously displayed. Even better, get a front page 
link to your program specifically. Of course, if you do, everyone will want one and if your page is like 
ours, it would soon be overrun with links to distance learning programs. If a student is looking for 
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distance learning opportunities, she is not going to search your whole site to see if you have them. 
Advertise on your own institutional home page. 

Another valuable Web resource is GradSchools.com, which has been a very good source of prospective 
graduate students for us and is not very expensive (free, actually). There is a growing number of sites, 
sponsored by university consortia (such as SREB), state agencies concerned with higher education (Texas 
Higher Education Coordinating Board), and institutions (University of Texas TeleCampus) which 
maintain Websites listing programs and courses available online. Generally, since the sponsors of these 
sites have a vested interest in the availability of distance learning resources, the listings are free. Be sure 
your courses get listed on appropriate sites of this type. 

A Final Thought 

Hang in there. Persevere. In the wise words of a colleague, “We know we are making progress when the 
problems we are having this semester aren’t the same ones we had last semester.” Just keep the needs of 
your students in the forefront, and you will prevail. 
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Introduction 

A key to providing quality support for online teaching and learning is the selection of and ongoing 
relationship with a courseware platform provider. Beyond staffing, this may be the most important and 
most expensive element of a distance education program. 

Course Design 

A courseware platform creates the parameters within which online courses must be designed. The 
platform functionality determines the way in which content is organized, the technologies which may be 
used for instruction, the communication formats available for faculty to student and student to student 
interactions and the ways in which students may be evaluated, among other things. In addition, the 
courseware platform determines the skills needed by faculty and staff, the ways in which faculty and staff 
must be trained, and the amount of staff support needed for course maintenance. 

Faculty and Student Satisfaction 

The quality of a courseware platform is, in many ways, a major determinant of the quality of the faculty 
and student experience. If a platform has the functionality needed by faculty and if students find the 
platform easy to navigate and use, both are more likely to have a successful online learning experience. If 
a platform is limiting, confusing, has bugs or other problems, fhistration levels rise and the confidence in 
online learning of one or both groups may be compromised. 

Resources, Options and Limitations 

Many courseware platform companies provide ancillary services such as technical support, course hosting 
and access to staff for platform customizations. This can bring valuable resources to an institution. Some 
courseware companies require the purchase of these services. This can be limiting if one or more of the 
services is poorly designed or managed. On the other hand, purchasing these services from other 
companies — if not available through the courseware provider or if the provider allows the organization to 
purchase these services elsewhere — can allow the organization to provide the best of each service. But, 
this may be expensive and/or difficult to manage. 

The UT TeleCampus 

The University of Texas System TeleCampus (UTTC) is a centralized service organization that provides 
support for the development and delivery of online degrees and courses for the 1 5 component campuses 
of the U.T. System. Founded in 1 998, the UTTC currently offers 7 master’s degrees, a series of general 
education courses, a bachelor’s completion program and several other higher education programs and 
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certificates. With an inventory of about 150 higher education courses, the UTTC supported the teaching 
and learning of approximately 100 faculty and 6000 student enrollments during the 2001/2002 academic 
year. 

The goals of the UTTC are to provide support services to distance learners, to expand the reach of the 
U.T. System within the State of Texas, to increase student enrollment at some U.T. System campuses, to 
extend outstanding programs to students beyond the originating U.T. System campuses and to use 
economies of scale to minimize the cost of distance learning efforts. 

Request for Offer and Platform Selection Process 



Vendor Questionnaire 

As the three-year contract with the UTTC’s first courseware platform provider came to a close, the UTTC 

staff to begin a process to select a new courseware platform provider. The UTTC staff included 36 

questions in the RFO to which courseware providers were asked to respond. Those questions are listed 

below. 

1 . Explain in detail the attributes of the courseware platform you propose. 

2. Describe the ways in which content can be entered into, deleted and modified within the 
courseware. 

3. Does the courseware have a built-in HTML editor or wizard for use in creating course content? 

4. Describe the ways in which multimedia elements may be integrated into the courseware. What 
are the abilities and limitations of the courseware regarding multimedia elements (e.g., Java, 
Shockwave, etc.)? 

5. How are interactive elements integrated into the courseware? What are the abilities and 
limitations of the courseware regarding interactive elements? 

6. Describe the courseware content organizational structure. 

7. Does the platform have the capability of linking or branching within the courseware? 

8. What components of the course, if any, may be modified while the course is being offered? Can 
faculty members make the modifications? 

9. Describe the course navigation abilities and flexibility. 

10. Describe the student assessment and evaluation attributes of the platform. Is branching from test 
responses available (feedback on responses, remediation, etc.)? Can evaluation elements be 
timed or dated? Can test pools be used? Can test questions be randomized? 

1 1 . Describe the student tracking abilities of the courseware platform. 

12. Is there a gradebook within the courseware? Describe the attributes of the gradebook. Will the 
gradebook link to evaluation elements within the courseware? Can the faculty member add 
scores from external evaluation elements? Can the gradebook design be modified while the 
course is being offered to students (addition or deletion of elements, weights of items, etc.)? 

13. Describe the method by which your system can be tied into our student information system. 
Specifically address your proposed method to automate the registration (granting of access) of 
students into courses via a batch process. 

14. Describe the different forms of course access (administrative, faculty, student, etc.). Is there a 
limitation to the number of each form of access per course? Can access be defined for each 
individual? (For example, can a faculty member determine which elements of faculty access will 
be allowed for a teaching assistant?) 

1 5. Does your system provide web-based conferencing? Does that conferencing support both 
synchronous chat and asynchronous threaded-discussions? Is that chat logged to an accessible 
file? 

1 6. Are glossary and references functions available? If so, are they searchable? 
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17. Can portions of a course be made available to potential students for review? (For example, can 
the syllabus be made available outside the secured course?) 

18. What is your corporate strategy and commitment to staying current with emerging technologies 
and standards? Please speak specifically to compliance with IMS and SCORM standards. 

19. How would you address training and technical support for TeleCampus staff? 

20. What type of technical support will be available to faculty and students? If a help desk or other 
technical support email or phone number is available, what are the days and times of availability? 

2 1 . Are faculty and student tutorials available? If so, describe. 

22. Describe the communication system and processes that will be put in place to facilitate the 
working relationship between the company and the TeleCampus. 

23. Describe in what ways your system allows for automation of student access to courses. How 
“open” is your system in terms of integrating back-office processes with our Student Information 
System? 

24. What is the process by which bugs or other technical problems are resolved? 

25. What would be the minimum required user configuration (processor, browser and version, etc.) 
for course access? 

26. Will course development be possible on PC and Macintosh computers? Will courses be available 
to students using Windows, Macintosh and Linux systems? 

27. How will you handle modifications/upgrades of the courseware platform? How frequent are 
modifications/upgrades? If existing courses are to be upgraded, how do you propose that 
happen? 

28. Describe your course hosting options and models (e.g., licensed to end-user, vendor-hosted, etc.). 
If hosting will be outsourced to another company, please name that company and describe the 
relationship between the two companies. 

29. How will the hosting system handle back-up and recovery? What will be the frequency of back- 
up? 

30. Do you maintain a development environment separate from your production environment? 
Describe the differences between the management, security and hosting of courses being 
developed and courses being offered to students. 

3 1 . Provide data on server downtime during the most recent 1 2-month period. 

32. How would you handle the issues of course conversion from the current platform (if necessary)? 

33. Please address our need for documentation and on-line help regarding the courseware. 

34. Please present a general timeframe for implementation of the relationship, including TeleCampus 
staff training, server set-up and conversion of existing courses. 

35. Please provide the names of three to five academic clients. Describe the length of your 
relationship and the number of courses each client is offering on the platform. Please provide 
contact information for those clients. Please be sure one of the clients is your most active 
academic client (i.e., the most courses or annual course offerings). 

36. Please provide any additional information that you believe is relevant. 

RFO Review 

After proposals were eliminated that did not meet the deadline or that were not complete, the proposals 
were reviewed internally and externally. Internal reviewers were the UTTC management staff. External 
reviewers represented campuses and included faculty, course production staff, distance education 
administrators and administrative leadership. All reviewers completed a courseware evaluation matrix 
that was based on the criteria for selection. 
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1 . 



Total cost (including start-up and per-student) as derived from information contained within the 
offer and clarified during negotiations. (35%) 

2. The quality of the response to the Vendor Questionnaire. (30%) 

3. Past performance of the vendor’s system in other, large scale “virtual universities.” (15%) 

4. Assessment of vendor strength and long-term viability. (10%) 

5. Proposed migration methodology of existing courses to vendor platform. ( 1 0%) 

Narrowing the Candidates 

When the initial reviews were complete, the results were tallied and provided to the UTTC management 
staff and RFO respondents were each offered an opportunity to present their proposal in a two-hour 
meeting. Copies of the proposals were provided to all interested UTTC staff who were also invited to 
participate. At the completion of the presentation cycle, the UTTC managers made their first vote to 
narrow the candidates. Two finalists were selected for additional review. 

Selecting a Final Candidate 

After the selection of the final candidates, UTTC managers called the institutional contacts listed by the 
vendors in response to question number 35 of the vendor questionnaire to pose a series of questions. 

1 . Which product is being used (release and version)? 

2. What is the size and mission of your institution (public, private, for-profit)? 

3. What is the number of courses developed on this platform (true web based or web-enhanced)? 

4. Do you host the product locally; does the vendor do the hosting? Or, third party hosting? 

5. What other platforms were considered before selecting this one? 

6. What were the reasons for the selection of this product? 

7. How long have you had the product in service? How long from contract to production? 

8. How has the level of service met your expectations, based on initial plans and promises? 

9. Were there any significant additional costs above what was initially contracted? 

1 0. How frequently have you had to call tech support? Are you satisfied with the timeliness and 
quality of the resolution to the calls? 

1 1 . What is your overall service and support satisfaction? 

12. Did you integrate the product with your student information system (SIS)? If yes, how 
successfully? 

13. What are best/least desirable features of the platform? 

1 4. All things considered, would you renew your contract with the company? 

15. Do you have any advice for us? 

In addition, requests were made to the two platform providers for access to the courseware. Blank 
courses on each of the two platforms were assigned to three staff in the curriculum department who each 
built all or part of a course on each platform. As a final step, all UTTC staff members met with 
representatives of each of the courseware platform companies for a half-day. The meetings each wrapped 
up with a discussion between the company representatives and the UTTC management staff regarding 
platform pricing. After review of the reference checks, presentations by curriculum staff and meetings 
with company representatives, a final candidate was selected and contract negotiations were begun. 

Summary 

In summary, the UTTC learned valuable lessons regarding platform provider selection. It is of primary 
importance that universities and colleges enter into the RFO process with a clear idea of what they are 
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seeking. They simply must know the products. The less they know, the more likely they are to make a 
mistake in their selection. 
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Introduction and Background 

With the expanding infrastructure for telecommunication in the world today, new applications for 
communications are emerging. To assure success in the workplace of tomorrow, it will be necessary for 
students to have collaboration and communication skills that utilize technology tools (CEO Forum, 1997; 
Schlechty, 1997). Videoconferencing, also referred to as teleconferencing, is a two-way method of 
communication between two or more groups in separate locations via audio, video, an^or computer 
systems. As a communicative technology, videoconferencing offers the potential to provide students 
relevant skills, connect distant or remote learners, and induce change. 

These communication elements are central to the mission statement at California State University San 
Marcos. The faculty is committed to transforming public education and has engaged in research that 
identifies effective teaching and learning strategies within distance learning environments that include 
videoconferencing (Hanor, Hollenbeck & Wendling, 1997; Hayden & Hanor, 2002). One simple 
defmition of distance learning is, “learning without the physical presence of the instructor” (Mantyla, 
1999, p. 4). As in distance learning, videoconferencing may separate teachers physically from learners. 
What is lost and what is gained in the process of separation? Teachers and administrators must confront 
issues such as these while they receive training in the design and application of appropriate uses of 
videoconferencing to support effective, efficient, and meaningful learning. 

As educators are exposed to new methods for instructional delivery, old analogies are often applied. 
“Educators tend to use new technologies within the context of their perceptions of instructional strategies” 
(Hayden, 1999). Videoconferencing experiences can be effectively integrated into constructivist learning 
environments. The results of a videoconferencing study completed in 1999 (Hayden) were used to guide 
the planning of videoconferencing seminars for a professional development project implemented in 
Southern California between May 2000 and May 2002. 

Improving Learning for All Students through Technology (ILAST) was one of thirty state funded 
professional development projects to improve K-12 teachers’ and administrators’ skills for applying 
technology to teaching and learning. Within the context of the ILAST partnership, a collaborative system 
for videoconferencing was established in which the San Diego County Office of Education provided a 
bridge for multi-point connections among four to five locations simultaneously. Using dial-in systems for 
connectivity, each site had two-way visual and audio access. While technical information can be provided 
for duplicating this system, the intent of this paper is to share the pedagogical research and discoveries 
that have been made as a result of these experiences with multi-point videoconferencing. 
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Research and Philosophical Foundations 



A study completed in 1999 (Hayden) used the Delphi process to collect data regarding connections 
between videoconferencing and instructional strategies that enable constructivist learning. A panel of 
experts was asked to draw from their experiences with education, technology, and videoconferencing to 
identify characteristics and critical support strategies necessary for videoconferencing in constructivist K- 
12 environments. The panel’s recommendations have provided a guide for technology trainers and 
professional development planners in implementing effective teacher training and support. 

Twenty characteristics of videoconferencing that support constructivist learning emerged in the study 
completed by Hayden (1999). These twenty characteristics are categorized into four themes in Table 1: 

(a) connections, (b) questioning, (c) learning, and (d) interaction. Within this list of twenty characteristics, 
ten characteristics of videoconferencing were rated higher than the other ten. By organizing the top ten 
characteristics by theme area, learning experiences using videoconferencing to support constructivist 
learning can be planned. Table 1 shows how a videoconferencing project can incorporate all ten of the top 
videoconferencing characteristics. 



Table 1. Planning An Activity Using The Top Ten Characteristics Of Videoconferencing That Support 
Constructivist Learning Environments (Hayden, 1999) 



Theme 


Characteristic 


Scenario: 


Connections 


Synchronous connections between 
students and primary sources such as 
experts and remote locations; involving 
multiple sites in activities. 


Students are assigned an authentic task 
that offers opportunities for planning 
connections outside the classroom with 
experts and primary sources such as 


Questioning 


Students develop and ask questions; they 
are in charge of their learning. 


museums. The students plan, develop 
and ask questions during a 


Learning 


Students present to remote partners using 
audio and video for communication. 


videoconference to collect information 
and gain an understanding of key 


Interaction 


Students work in groups on authentic 
activities with remote sites. This involves 
videoconferencing with remote sites and 
use of an online suite of tools to support 
videoconferencing activities. 


concepts. The students use a suite of 
online tools as follow-up to the initial 
interaction and then present their 
findings “live” to remote partners 
concluding their research. 



Development of Strategies Based on Research 

The format for the ILAST videoconferences was developed using the effective constructivist practices 
and interactive techniques identified by Hayden (1999) in four categories. 

Connections 

Participants contributed to the selection of topics and attended videoconferences at one of five locations 
convenient to their school or home. Experts and/presenters shared ideas, projects, resources, and provided 
opportunities for participants to raise questions, respond to content and interact with other participants or 
materials during the seminars. During some seminars, additional sites were invited to connect and 
participate in order to provide access to primary sources of information located remotely. Sometimes a 
discussant summarized key issues, suggested connections, identified emerging themes and trends, or 
pointed out possible areas for future research or collaboration. Participants from all of the sites were 
invited to contribute to group discussions and ask questions during designated times of each seminar. 
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Questioning 



A key component of each videoconference seminar was question-and-answer periods encouraging 
participants to clarity and confirm concepts. Participants frequently were divided into groups and worked 
collaboratively at each site and among sites to raise questions, identify critical issues and reflect on the 
session with guidance from appointed facilitators. The use of laptops at each site enabled individual or 
small group inquiry into designated software or online resources appropriate to the topic of study. Each 
site shared questions, responses, and reflections and continued to build knowledge on the topics following 
the interactive seminars. 

Learning 

“When teachers are learning to integrate technology into their classrooms, the most important staff- 
development features include opportunities to explore, reflect, collaborate with peers, work on authentic 
learning tasks, and engage in hands-on, active learning” (Sandholtz, Ringstaff, & Dwyer, 1997, p. 142). 

Participants were engaged in the process as well as the content of the seminars. Participants contributed 
to the design, implementation and assessment of videoconferences and practiced methods and strategies 
that had been modeled at previous videoconferences. This contributed to understanding the 
videoconferencing components that could be applied to their own classrooms. Presenters were located at 
different sites and sometimes at more than one site to distribute the expertise and collaboration among the 
partner locations. Whenever possible participants were invited to share their expertise at future seminars 
and reflect on their learning experiences with their mentors. 

Interaction 

“Using real-time interaction through computer text-chat or videoconferencing programs create a 
“telepresence” between students and other people around the world" (Harris, 1998, p. 8). 

After each videoconference, opportunities were structured that enabled participants to contact experts, 
contribute to discussion threads in an online bulletin board, and visit resources posted on the ILAST web 
page (http://www.csusm.edu/ilast). While some issues and topics were derived from the competency- 
based curriculum for ELAST, participants played a critical role in determining topics that they considered 
valuable. This enabled topics such as “Addressing the Needs of English Language Learners”, an issue at 
many partner schools in Southern California, to be included. A search identified several local experts and 
examples of best practices that had been implemented by nearby schools, including ILAST partner 
schools. The experts not only shared their experiences and projects, but also invited participants to visit 
their schools. They encouraged continued contact through email. 

Conclusions 

Each of the eleven ILAST videoconference seminars provided an opportunity to reflect and evaluate the 
effectiveness of the conference formats. Participants completed evaluation forms at the end of each 
seminar. The ILAST leadership team analyzed and reviewed participant comments and reflections. 
Lessons were learned regarding selection of format and content to best meet program and learner needs. 
The leadership team found the following: 

In response to technical analysis of connections, participants were fairly accepting of temporary visual 
“disconnects,” but much more limited in their tolerance for auditory problems. Participants appreciated 
strongly presented content with explicit links to standards and curricula and visuals to support key 
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concepts. ILAST seminars used video, document cameras, presentation software and web projection to 
broadcast visual images. Each of these media offered support for visual learning but the technology 
offered challenges through varying bandwidth, and audio/video technical difficulties. When technical 
difficulties were experienced, the participants were very critical and felt as though their time was not 
productive. Backup plans, as recommended in the findings by Hayden (1999), were essential so that each 
site could proceed independently when technical difficulties were experienced. One difficulty was that 
experts in the content area could not be at all sites and site facilitators lacked confidence in assuming the 
role of an expert. 

Participants identified certain criteria for determining their satisfaction with the ability to raise questions 
and interact with content as well as people. They questioned the efficiency and effectiveness of 
videoconferencing in comparison to other learning formats. Although each seminar had scheduled times 
and activities for interaction with content, experts and other practitioners, the ratings for the evaluation 
question: “rate the ability with which you were able to participate in the videoconference” ranged from 
low (47 % rated a 1) to high (57% rated 5) on a five-point Likert scale. 

The category of learning was evaluated in the survey by asking several questions related to content, 
application and satisfaction with the seminar content. Opportunities for questioning and interaction during 
each conference were thought to provide an opportunity to engage and motivate participants. Evaluation 
responses to learning were shared through reflections during the conference and throu^ posting and 
emails to instructors and experts following the conference. The area of content was generally rated high 
when the technology worked and participants could see and hear the transmission. When participants 
were asked to identify “the best thing about the videoconference,” they always overwhelmingly 
responded that the presenters were most important, followed by the area of content. 

Preparing our presenters for the interactive format of the seminars presented another challenge. 

Guidelines were prepared and sent to presenters ahead of time to help them prepare for interactive 
formats, but this issue continued to be a challenge. Most of the experts that were scheduled as presenters, 
planned to present information in a “lecture” format in their content proposals in spite of the 
recommendations. Each time this occurred, the participants responded in their evaluations by stating that 
they needed more interaction and “less lecture.” The planning team found it helpful to post pre-seminar 
information on a web page prior to the videoconference to prepare participants in advance for the 
seminar. This activity encouraged increased interaction with the content, with the expert and/or materials 
being presented during the videoconference. 

Implications and Future Work 

The ILAST leadership team has further investigated the findings from the Delphi Study and has found the 
characteristics recommended by the study to be valid. As the second year of professional development 
draws to a close, the evaluations for all videoconferences are being continually analyzed to document the 
successes and challenges. The leadership team plans to further the study and identification of effective 
constructivist practices for interactive videoconferencing for professional development. The support 
strategies for videoconferencing have been distributed to the schools in the ILAST partnership. This study 
will help inform decisions made by schools and districts planning to acquire videoconferencing 
equipment and implement this new media in student learning activities. 
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Building a Highly Accessible Web-Based Training Course 
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Mike Rafferty 
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Introduction: Distance Education and AccessibUity 

In theory, distance education is ideally suited to the needs of people who are not able to attend traditional 
learning environments. People with disabilities, travelers, or those who live in remote areas can find it 
difficult to attend traditional classes and are thus perfect candidates to participate in Internet-based 
distance learning. 

Unfortunately, distance education courses are not always accessible to all of the students who wish to take 
them. Barriers exist for a variety of reasons. To eliminate these barriers. Internet-based distance 
education courses should be designed with an emphasis on accessibility. Design for accessibility takes 
into account the abilities of the student, as well as the abilities of the student’s computer hardware and 
software. 

Design for accessibility involves a number of different techniques, some of which are quite easy to 
follow. Two distinct areas of focus are instructional design and web site design. By structuring course 
content in an appropriate manner, accessibility can be maximized for people having various learning 
styles, cultural backgrounds and abilities. Appropriate use of technology and careful web site design are 
the other key factors that enhance accessibility. 

While improving accessibility is crucial to the success of distance education courses, it is important to 
maintain student interest. Graphics, interesting and interactive course content, and creativity can be 
combined with accessibility techniques to form an engaging and accessible training course. 

Understanding Accessibility 

Many people experience difficulty using the Internet and computers. Difficulties can arise due to 
disabilities and limitations of computer hardware and software. Developers of courseware can take steps, 
some of which are quite simple, that will greatly improve accessibility. The initial, easy steps taken to 
improve accessibility will result in benefits for all users. 

The more difficult and specialized techniques require judgment and compromise. It is possible that in 
attending to the needs of a particular type of disability, content might become less accessible to people 
with different disabilities. Maximizing accessibility involves following established guidelines as well as 
determining the appropriate balance to strike between the needs of the users. 

Abilities can vary from person to person. Some people have combinations of disabilities, or disabilities 
of varying severity. The following is a list of disabilities that affect people’s abilities to access 
computers. 

❖ Blindness: a substantial loss of vision in both eyes 

❖ Low Vision: vision that is not sharp, or is sharp only in portions of the field of vision 

❖ Color Blindness: a lack of sensitivity to some colors 
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*t* Deafness: a substantial impairment of hearing in both ears 

❖ Hard of Hearing: mild to moderate hearing impairment 

❖ Motor Disabilities: various conditions that affect the ability to move or to control movement 

❖ Cognitive & Neurological Difficulties: dyslexia, dyscalculia, attention deficit disorder, memory 
impairments, mental health disabilities 

❖ Seizure disorders 

In addition to disabilities, a person might have trouble with accessibility due to the software and/or 
hardware they’re using. It’s helpful to understand the various reasons why a person could be unable or 
unwilling to install a particular type or version of hardware or software on a computer. Some reasons are 
listed below. 

❖ They do not have the expertise to install it 

❖ They cannot afford to purchase it or have it installed. 

❖ They cannot install it because they use a shared or public computer 

❖ The computer they use may be too old to support it 

❖ They may be using a Web appliance or service that does not allow changes 

❖ They may be using an operating system that will not support it 

❖ They may not be willing to wait through the download 

❖ They may have special needs which restrict their ability to use some browsers or software 
Put Yourself in Someone Else’s Shoes 

Experiencing web pages through the browsers of other people can help in understanding the accessibility 
needs of others. Many people do not have the latest versions of the Netscape Navigator or Internet 
Explorer browsers. And for various reasons some people have their browsers configured to not show 
graphics or to not run JavaScript. 

In addition, some people use alternate browsers such as Opera, which has many accessibility features, or 
the text-only Lynx browser. To further complicate things, some people with disabilities use screen 
readers or self-voicing browsers, which are software programs that use a speech synthesizer to read aloud 
the text on a page. 

A courseware developer must be aware of this and strive to eliminate barriers that arise due to the 
differences in browsing setups. Testing your web pages in various browsers and screen readers, along 
with following established accessibility guidelines will enable you to improve the accessibility of your 
courseware. 



Improving Accessibility Through Instructional Design 

Educational content is the cornerstone of a course and its design can have a significant impact on the 
accessibility of the material. All the fancy spinning graphics in the world will never replace interesting 
and engaging material delivered in a creative and interactive way. Accessibility can be improved, and 
web-based learning environments can be made to be more interesting if a few basic considerations are 
addressed in the context of Instructional Design. 

Know Your Students 

By knowing the motivations and expectations of your students, you can design a course that best suits 
them. You can also anticipate barriers and improve accessibility by understanding their needs and 
abilities 
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Manage the Subject Matter 

Reformat traditional educational text to make it more readable on the Internet by: 

❖ “Bracketing” the material by adding introductions and conclusions 

❖ Using sub-headings to break-up large chunks of text 

❖ Knowing how much information is enough 

❖ Keeping paragraphs brief and using bulleted lists 

❖ Adding internal hyperlinks to emphasize key points 

Enhance the Learning Environment for Adult Learners 



This is done by: 

♦> Clearly defining learning outcomes. 

❖ Designing assessments to reinforce and assess mastery of objectives. 

❖ Increasing interactivity 

❖ Build in student feedback, such as quizzes 

❖ Allow students to have control over their learning experience by offering choices on how 
to navigate through the course 

❖ Use multiple interactions 

Create an Esthetic Experience That Enhances the Learning Environment 

Choose an easy to read font and use plenty of white space to aid in reading ease. Text should be placed 
on a background that allows for high contrast and is visually appealing. The judicious use of graphics can 
be an excellent tool in facilitating learning for the visual learner. 

Improving Accessibility Through Web Site Design 

There are numerous organizations promoting accessibility guidelines for web site design. Foremost is the 
Web Accessibility Initiative (WAI), a division of the World Wide Web Consortium (W3C). The primary 
reference resources for developing accessible HTML Web pages are the WAI Web Content Accessibility 
Guidelines 1.0 (http://www.w3.org/TR/WCAG10/) and Checklist of Checkpoints for Web Content 
Accessibility Guidelines 1.0 (http://www.w3.org/TR/WCAG10/full-checklist.html). In addition to 
guidelines, the WAI provides detailed examples and reference materials on web accessibility. 

A few highlights of accessibility guidelines are: 

❖ Use clear and consistent navigation menus 

❖ Use alt tags to provide text explanations for graphics 

❖ Use relative rather than absolute sizing for tables and fonts 

❖ Provide for keyboard navigation through use of TABINDEX 

❖ Avoid the use of tables for page layout 

❖ If tables are used for page layout, they should degrade gracefully if the user agent (browser) 
cannot interpret tables 

❖ Use text-only pages as a last resort 

❖ Use structured HTML for text formatting, rather than outdated commands such as Bold and 
Italic 

❖ Do not rely on color alone to communicate meaning. 
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❖ Use high contrast colors 

❖ Test web pages using a number of different browsers and window sizes 

❖ Validation and testing of web pages must include human review 

Note that the guidelines do not prohibit the use of graphics and other visually interesting elements. In 
fact, by relying too heavily on plain blocks of text, a web page can be made less accessible to visual 
learners. For this reason, it is important to incorporate graphics and color onto the web page. 

Conclusion 

By carefully applying instructional design and web design techniques, accessible training courses can be 
created that benefit all students. A side effect of designing for accessibility is that simple, straightforward 
software is used, which reduces the cost and eases the process of developing and maintaining the course. 
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This paper presents results of a qualitative analysis of mentor support for distant adult learners in a web- 
based learning environment. It combines two sources of data: emails mentors sent to their students and 
mentors’ reflections about mentoring students in a Master of Arts of Learning and Technology program at 
Western Governors University (WGU). By examining the data from these sources, we gained insights 
into our mentoring practices and hoped to find useful strategies for helping students progress in a timely 
and successful manner through our program. 

Western Governors University implements a model that separates assessment from instruction, which 
means that its faculty, the mentors, do not directly deliver instruction. Students take online courses 
offered by other universities, known as Education Providers. The responsibilities of the WGU mentors are 
to provide academic guidance, advising, and tutoring to WGU students throughout their programs. 
Mentoring activities range from designing a preferred path, preparing students for assessments, reviewing 
portfolio projects, and serving as Chair for students’ Capstone Committees. Mentoring is conducted via 
email, listserv, and telephone. The main communication charmel between students and mentors is email. 

The MET program at WGU consists of five domains and 17 assessments. Students’ progress is measured 
by the number of assessments they have passed. Therefore, it is important for a mentor to motivate 
students to take assessments after the students have completed the recommended learning opportunities. 
Our experience tells us that the first 4 to 6 months of a student’s program are crucial for retention and 
eventual graduation. However, students enter our program on a monthly basis, while most of our online 
courses are semester-based. In addition, students begin with different skill levels, necessitating 
individualized learning. Thus, we were particularly interested in finding out what activities our most 
rapidly progressing students were engaged in during the first few months of their programs. Emails 
provided the best window through which we could view those activities. 

Mentoring students at a distance in a competency-based online environment is new and there are few 
models to rely on for support. Mentors employ their own strategies to help students progress. After more 
than two years’ practice, each mentor has accumulated some thoughts and tips for mentoring adult 
learners at a distance. This reflective part of the paper will identify and share our best practices for 
helping future students progress. Results may be valuable for educators engaged in research on, or the 
practice of, mentoring in a distance-learning environment. 
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Methodology 



Participants 

There were two types of participants for this analysis: the three researchers/mentors as direct participants 
and 30 students as indirect participants. The students were included only because they were the email 
recipients and correspondents of the participating mentors. Each mentor’s students were ranked by their 
spe^ of progress based on the number of assessments completed and number of months a student had 
been in the program. The top ten from each group were selected. Two students were removed from the 
list due to the loss of their first six months’ emails. Therefore, only emails sent to 28 students were 
selected for analysis. 

Collection of Data 

1 . Each of the researchers/mentors collected her/his own emails from Microsoft Outlook Sent Box. 
Five to six months of emails sent at the beginning of each student’s program were chosen for 
analysis. 

2. The researchers wrote separate reflections about their mentoring practices in response to four 
guiding questions (see Results section below). 

Data Analysis 

Through a rough check of emails sent to students, a tentative set of categories was developed. The set 
was fiirther discussed and revised. The initial set of 26 categories was detailed and specific for easy 
coding and control of reliability. The categories were entered in a spreadsheet for coding. During the 
coding process, categories were added or deleted as necessary. The initial categories were grouped into 
14 major categories in the analysis process. The final set of categories was summarized in Figure 1 
below. 

The reflections of the researchers/mentors were synthesized and summarized in the Results section, 
below. 



Results 

The three coding sheets were merged and summarized in Figure 1 . 

As Figure 1 shows, mentors sent more emails that were related to assessments and portfolio projects. It 
seems that early in their programs, these students were already engaged in these activities. Our initial set 
of categories shows that mentors were either announcing assessment results, providing feedback on 
portfolio activities, scheduling for competency review, or answering questions regarding assessments or 
portfolio activities. 

The next largest group of emails fell into the categories of resources, encouragement, and phone calls. 
Resources here means mentors were sending students various resources for their domain assessments or 
domain projects. Phone calls here means scheduling phone calls. Mentors always review competencies 
with students on the phone when they are ready for a domain assessment and scheduling of those phone 
calls is usually via email. Thus, these categories fiirther point to the fact that the selected students had 
dived right into the core activities of the Master’s program early in their WGU studies. 

The comparatively large volume of encouraging emails from mentors indicates that encouragement or 
praise was a good motivator, as discussed in the following Mentoring Strategies section. 
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Although it is too early to draw any conclusions from this preliminary email analysis, results seem to 
suggest that it might make a difference if a mentor encourages students to attempt at least one assessment 
early in their program. The experience of taking an assessment could reduce testing anxiety and/or 
increase self-esteem and confidence. 
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Figure 1. Emails sent to students 



Mentoring Strategies 

The researchersVmentors’ reflections reveal similarities and differences in mentoring practice. Results 
are summarized based on four guiding questions, included below. 

1. What challenges have we faced in moving our adult learners through a competency-based, online 
learning model? 

The first challenge was to help students see the importance of structuring their learning time and setting 
aside a specific number of hours per week for their studies. An additional challenge was helping students 
realize their own ability to pass our domain assessments or helping them cope with failure in an 
assessment. 

Another challenge was to help students, who entered the program with inadequate writing and/or 
technology skills. Unlike traditional universities, WGU hires outside graders to score students’ portfolio 
projects using rubrics for consistent quality control. As a result, the writing requirement is rigorous and 
daunting, causing some students to tend to procrastinate rather than to revise their portfolio projects and 
move forward. Occasionally, we find students who desire to learn how to integrate technology into their 
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teaching curriculum, but who come with limited technology skills and understanding. These students 
may become very frustrated when they encounter technical difficulties in their first online courses. As a 
result, their frustration easily builds and their progress wanes. 



2. How do we motivate students to progress in this web-based and non-traditional learning 
environment? 



Students enjoy the freedom of the “any time and any place” feature of online classes, but they tend to get 
frustrated without face-to-face interaction, as well as examples and models to rely on. They feel online 
learning is similar to independent study and much harder than learning in a traditional classroom. 

Mentors try to build strong rapport via email and phone calls with their students so that learning in online 
courses is less lonely and intimidating, or frustrating. While providing the necessary academic support, 
mentors also try to psychologically prepare students at the very beginning of their program for failures in 
assessments or portfolio activities by emphasizing the feet that a competency-based model of learning 
naturally entails multiple attempts at assessments to reach the competencies. Students soon learn that they 
can turn to their mentors at any time for academic and emotional support, and for solving problems, thus 
giving them the courage to progress (Brookfield, 1986). 

3. What strategies do we use to help new students have a good start in the competency-based 



Mentors have learned from experience that a good understanding of the program ensures student success. 
Before a student starts her/his program, a mentor makes a detailed analysis of the student’s background 
knowledge, expectations of the program, and future career interests. Based on the information obtained 
from the skill survey and the pre-assessment that a new student takes as required items for admission, the 
mentor gains a good understanding of a student before the first extended phone interview with her/him for 
structuring the Academic Action Plan. During this phone interview, mentors use the strategy of an 
advance organizer (Ausubel, 1968) to give the student a good overview of the program, detailing how 
each domain project can be integrated into the final capstone. Most important of all, the mentor tries to 
relate the program projects to the student’s career interest, and help the student identify some instructional 
and learning needs in the student’s environment as topics on which to build the capstone. This initial 
discussion of the final project and the possible relation of the capstone to their work environment is so 
crucial that students immediately see the meaning and the benefit of pursuing the program. This 
discussion also coincides with many students’ expectation that they will be able to work on a project in a 
competency-based program to solve practical problems using a variety of technologies. 

4. What strategies do we use to keep them progressing smoothly in the program? 

In a distance-learning environment, we found that students really need immediate feedback to motivate 
them to progress. We tried to provide feedback within 24 hours or at least 48 hours. Praising students is 
also a great motivator; some students really need that encouragement according to the Learning 
Orientation Model (discussed below) and androgogical principles (Islam, 2002), which advocate placing 
learners in situations that promote their positive self esteem and show respect for the individual learner. 

Understanding students’ learning orientations has made mentoring more effective. According to the 
Learning Orientation Questionnaire (The Training Place, 2001), learners range through a continuum of 
transforming, performing, or conforming learners to resistant learners at the far end of the scale. An 
important role of a WGU mentor is to adjust her/his level of guidance and support to accommodate 
different learning orientations. We noticed that learners could move in and out of one orientation in 
response to negative or positive responses, conditions, results, and experiences (The Training Place, 
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2001). Thus, we tried to create a conducive environment for students at the very beginning of their 
program in an attempt to help them move more toward the performing or transforming end of the 
continuum of learning orientations to take more control of their learning. 

We also tried to build strong relations with students through flexible availability for phone appointments 
and demonstrations of caring. A strategy adopted by an individual mentor is to require weekly reports 
from students in an attempt to make students set their own goals and deadlines. This strategy, which is 
used in addition to a monthly phone call, has proven effective by the mentor. 
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Campus Outreach Efforts: 

Moving Beyond Lecture Notes to “Interactive” Online Instruction 



Susan L. Jones, Ph.D. 

Assistant Professor/Co-Principal Investigator 
U.S. Dept of Education, PT3 Grant 

Introduction and Campus Outreach Efforts 

Standards, standards and more standards combined with a rapidly changing technological age have 
provided faculty with new, exciting, and often challenging obstacles to overcome. A $ 1 .5 million dollar 
U.S. Department of Education, Preparing Tomorrow’s Teachers to Use Technology (PT3) grant has 
provided Southwest Missouri State University (SMSU) faculty with state-of-the-art technology, release 
time, outside consultants, a full-time in-house consultant, opportunities to work with the St. Louis Science 
Center, and technology-sawy partners at three technology-rich K-12 public school districts in the State of 
Missouri. One of the objectives of the grant was not only to provide faculty with state-of-the-art 
technology that they did not have before but also to change the fact that new teacher graduates at SMSU 
were not incorporating best- practice uses of technology into their own K-12 teaching strategies due to a 
lack of “modeling” by faculty. At SMSU, the PT3 grant gives faculty members money to spend on 
technology of their choice that they will use in their own classroom. To further support the efforts the 
University has put in place: a Vice President for Information Services, a high speed data backbone with a 
domain structure; a fiber-based video backbone for Interactive Television; a bridge connecting the two 
networks; 30 separate servers for various academic needs; a campus-wide email system; an extensive 
web-based information system; a faculty development computer laboratory; a cadre of computer user 
support specialists; on-going software training courses for faculty; an instructional designer; a media 
distribution system in a newly constructed $26 million dollar Library Information-Annex; three campus- 
wide microcomputer labs and technology-based workshops for faculty/staff. Faculty now are modeling 
the use of their choice of technology in the classroom and many of them are now using the available 
online course management system. At SMSU, the course management system. Blackboard Courseinfo. is 
used on campus either to deliver a course or it is used as a web-supported system. Regardless, this in 
itself has brought about new problems. The biggest problem is that faculty are using the course 
management system to merely upload their daily “text-based” lecture notes, students are downloading 
them and/or printing them and reading the documents. Faculty members and students were/are missing 
out on the unique characteristics of the web that should be used. As part of the PT3 grant, there is 
currently an ongoing campus- wide outreach effort via web-based training modules and face-to-face with 
faculty using blended technologies to try and get faculty to move beyond posting lecture notes to creating 
interactive online instruction. In an effort to model for faculty how they can utilize the web, the following 
section will identify the unique characteristics that make teaching via the web different from teaching a 
face-to-face course. 



Unique Characteristics That Make Teaching Via the Weh 
Different From Teaching in a Face-to-Face Course 

The Internet and World Wide Web (WWW) are often used interchangeably though quite different. From 
the very beginning, as a government experiment the Internet was and has provided strong transparent 
connections between military and research institutions. Today, the Internet links computers together 
everywhere in the world. The WWW however is not the same. The WWW is accessed through browsers 
such as Netscape and Internet Explorer. The web uses “hypertext” as a way of interacting with users. 
H)q)ertext, can be read, searched, edited and even stored. Its unique features allow connections to be 
made within the body of the text to other documents, graphics, photos, video clips, audio clips, etc. In 
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Perelman’s (1992) book entitled. School’s Out: Hyperleaming, the New Technology and the End of 
Education, Perelman talks about hyperleaming which will allow students to learn through multisensory 
delivery modes at any time and any location around the world. While the WWW is still limited in its 
capabilities due to limited search engines, technological standardization, infrastmcture, and accessibility 
faculty members at SMSU have been trained how to utilized blended technologies that incorporate 
multimedia into their projects that get at the unique characteristics that the WWW provides. As the 
Internet and the WWW evolves Perelman’s vision may not be too far from becoming reality. At SMSU, 
faculty members across campus have had to confront the unique characteristics that make teaching via the 
web different than teaching face-to-face. Although each faculty member chose to purchase their own 
specific technology with PT3 grant money to help them meet their students needs, the following unique 
characteristics were found necessary to address by those using the WWW that were different from 
teaching in a face-to-face course. These include acknowledging and addressing: 

4X3 Aspect Ratio 

The computer has an aspect ratio, like that of instructional television. The aspect ratio is the vertical and 
horizontal ratio of a graphic in any presentation format. In essence, this is how the computer views 
information, people, graphics, and objects. Consequently, many faculty believed that they could take 
their overhead transparencies and scan them into the computer and then upload them into Blackboard 
Courseinfo. so that students would have access to them and could either read and/or download them when 
they wanted to. Understanding what the aspect ratio is and how it affects course material was important 
to faculty when they realized that in scanning their material and viewing it on a computer, they had 
cropped off the top and bottom portion of their material and it couldn’t be seen without scrolling up and 
down. Also, they noticed that they had not used up the complete width of the computer screen that was 
available, whereby making concepts and ideas less clear when scanned in in the inappropriate format. 

Graphic Design Elements for the WWW 

There are special graphic design elements that must be considered when creating and placing course 
material on the WWW for students to access and/or link to. For example, most faculty at SMSU use the 
Microsoft Office Suite software. PowerPoint is a favorite among faculty. PowerPoint, or even other 
similar software that is used to create course material for the WWW, cannot do everything for you so that 
your “slides” portray the use of good graphic design principles. Therefore, many faculty were taught 
some basic graphic design principles. For example, it is important that on any slide, your slide should 
contain words that are balanced and any graphic representation of the words on the computer screen 
should be balanced. In doing so, good graphic design elements also dictate that a lot of white space be 
left around the graphic and text should not be placed too close to the graphic so it is easier to read. Also, 
the total number of words on any “slide” should not exceed 20-25 words. Fewer words are preferable. 
Prompts and cues should be used by underlining, using italics, bold, color or all CAPITAL LETTERS. 
However, do not use too many of these at one time or confusion will be created rather than clarity and 
emphasis. Color is good to use, but limit color should be limited to 2-3 per page. Color should also be 
checked to make sure that the color rendition is good on various computers. Likewise, limit the number 
of font styles to three per page to avoid further confusion. Text must be checked for legibility. A good 
rule of thumb is to use at least a 24 point sans serif font. If possible, all text should be flush left, in upper 
and lower case letters for easier reading (Cyrs, Conway, 1997). 

Visualization 

Visualization is the ability to take abstract concepts and ideas and turn them into concrete representations 
using graphics, pictures, and video clips. Visualization is particularly important because “Well-conceived 
and rendered visuals help your audience understand and retain information” (Wileman, 1993, p.3). Many 
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faculty in the traditional classroom use overhead transparencies with primarily small-textbook like print; 
and/or write on the chalk and/or white board using primarily text. Some faculty use the document camera 
and data projector to project their overhead transparencies directly or print out a copy of their overhead 
transparency onto copy paper and then use the document camera and data projector simultaneously to 
project onto the screen. In working with faculty and pointing out the benefits of capitalizing upon the 
WWW’s visual ability (though if overused, download time away from campus T1 lines becomes much 
slower) most faculty at SMSU began incorporating more graphics, video clips, digital photographs, word 
pictures, and so forth, into their courses. In addition many faculty required their students’ to create visual 
multimedia presentations, although time consuming. Many faculty used software programs such as 
Inspiration, Mathematica, Powerpoint, Excel, etc. to create their visuals and then uploaded them into 
Blackboard, Courseinfo. These visuals were created to represent concrete facts, directions, processes, bits 
of data, comparative data, data recorded over time, organizational structures, places, chronologies, 
generalizations, theories, and feelings or attitudes. Although no specific quantative data was collected, 
most faculty expressed that they did feel that the visuals helped theirs students in learning, thinking and 
solving problems. 

Unlimited Access to Information Anytime/Anywhere 

Another unique characteristic of the WWW is the ability to create course material and post it online for 
students to access anytime and anywhere. In creating online course material, SMSU faculty, like Driscoll 
(1997) have found that there are two broad categories of online based teaching. The first type includes 
text only material that may include such things as email, software downloading, and utilizing bulletin 
boards. Email traditionally allows students to send and receive messages and send and return messages. 
Software downloading has traditionally meant that students could receive attachments of information and 
it could be download^ to their own computer for reading/printing. The second type of online-based 
teaching utilizes multimedia. Faculty members at SMSU were trained, by modeling, what a self-directed, 
self-paced learning environment on the web was. For example, in one particular online module, faculty 
were trained on how to use Blackboard by using multimedia to communicate the content to them. 
Interactions with the training material were conducted by the trainer and controlled as the faculty member 
advanced through the training module utilizing the anytime/anywhere concept. Further one-on-one 
explorations explained how instruction and learning could take place at anytime and anyplace, given that 
the students had proper access to equipment. As will be shown in the actual presentation during the 
conference, several faculty are still in the process of redesigning and uploading their course material into 
this type of virtual classroom to blend hypermedia, hypertext documents, on-line quizzes, modules of 
multimedia material, email, virtual classroom, discussion board, and group projects via Blackboard 
Courseinfo. In doing so, the primary role of the instructor shifted to guiding students to resources, 
assessing learning outcomes and providing constant feedback. Many of the faculty have chosen specific 
strategies that create an “interactive” learning environment for their students, even at a distance. Some of 
these include: brainstorming sessions, case studies, character dialogue, threaded discussion, 
demonstration, scavenger hunts, webquests, virtual field trips, group projects, interviews, learning dyads, 
interactive lecturettes, modeling, online office hours, questioning wrap-up, reaction panels, role playing, 
scenarios, and surveying. At SMSU, not all faculty are working within the second type of online-based 
teaching. In fact, there are several faculty who are still only comfortable with the first type of online 
based teaching as aforementioned. Likewise, there are several who are in the transition from moving 
from the first type to the second, as well as many combinations along the continuum. 

Conclusion 

Regardless, of where one fits on the continuum, many individuals believe that the WWW has the greatest 
potential for the future that provides educators with a way to organize and present instruction 
synchronously as well as asynchronously. The WWW is capable of providing educators and students 
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with print, video, audio, graphics, digital photographs, and data in a variety of formats with immediate 
access to information from sources all over the world. Regardless of the course management system that 
you are using (Blackboard, Web Course-In-A-Box, etc.) teaching on the WWW requires educators to 
redesign their courses with some adjustments made in how they prepare and present their course material, 
teaching pedagogy, and, considering course planning and preparation time; faculty and student hardware 
and software needs; security issues; costs; and student control. 
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Helping Faculty Develop Online Courses 
at a Distance 

Jahna Kahrhoff, Director 
Academic Distance Learning Center 
Webster University 

Qian Li, Instructional Designer 
Academic Distance Learning Center 
Webster University 

Introduction 

Founded in St. Louis, Missouri in 1915, Webster University is a private, multi-campus, international 
institution offering undergraduate and graduate degrees in Ae liberal arts, communications, fine arts, 
education and business and technology. Today, Webster has grown to 100 campuses in the United States, 
Europe and Asia. Webster’s online initiatives started in 1998 with launch of the Online MBA. Three 
other online programs followed; Master of Arts in Teaching (MAT) (1999), Web Development Certificate 
(1999), and MS in Business and Organizational Security Management (SECR) (2002). The Academic 
Distance Learning Center (ADLC) was established in 200 1 with a mission to create a quality distance 
learning experience for Webster University students and faculty by providing leadership, direction, and a 
range of support services for distance learning initiatives. The Center provides support in five major 
areas: program development, course development, WebCT administration, student support, and over all 
leadership and expertise in distance learning. The Distance Learning Center employs a comprehensive 
support team, including an instructional designer, course developer, course trainer, administrative 
coordinator, and student assistants. 

In the area of course development, the Center’s staff assist individual faculty (both full-time and adjunct) 
in the development and delivery of online courses by offering training, instructional design, course 
construction, and ongoing technical support. Because of Webster’s unique structure, online faculty are 
located all across the U.S. and working with them collaboratively presents an ongoing challenge to the 
ADLC staff. The ADLC staff have developed a successful process for working with faculty regardless of 
their location, using the Internet, email, phone, print materials, and more to bridge the geographic distance 
without compromising quality. Three general strategies are used to accomplish this goal: 

1. Division of Labor: The ADLC’s instructional designer, course trainer, and course developer 
divide and share responsibilities in order to meet the individual needs of faculty. 

2. The Course Development Process: Starting with a comprehensive needs assessment, the ADLC 
staff guide Webster’s online faculty through a step-by-step development process. A unique 
timeline and training process is customized to match each faculty member’s needs and style. In 
addition, specially developed work guides and online resources help the faculty through each step 
in the development process. 

3. Maintaining Contact: The ADLC staff initiating and maintaining frequent communication with 
faculty to help them stay on track. 

Meeting Challenges of Online Instructional Design 

Assisting Webster University faculty with the development of quality online courses is complicated by 
two challenges. The first is to identify an instructional design process that will help faculty move easily 
from the traditional classroom to the online environment in a timely manner. The second is to work 
collaboratively with faculty who are located all over the country. From our needs assessment forms 
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(which are filled out by new online instructors), we have identified the following challenges that faculty 
face when designing an online course: 1) lack of knowledge on how to transition from a traditional course 
design to an online course design; 2) lack of technical competence and/or experience in using online 
course management tools (WebCT); and 3) limited time available for designing an online course. 

To find an instructional design model that would be appropriate for online courses, we looked at several 
established instructional design models, specifically the ADDLE model and Dick’s LSD model (Figure 1). 
Both of these models are traditional and systematic models for designing classroom instruction and they 
can be used for designing online courses. However, we have found that they do not specifically meet our 
needs. First, these models are time-consuming. In addition, both models do not address the specific need 
of converting traditional classroom instruction into online instruction. 



ADDLE Model (Richey & Seels, 1994) 

The ADDIE model represents five stages of course 

design: 

1. Analysis - identify the instructional problems, 
assess needs of learners, and identify topics of 
instruction. 

2. Design - write course objectives, select 
instructional strategies, and identify assessment 
methods. 

3. Develop - select course materials and develop 
instructional materials. 

4. Implement - deliver instruction. 

5. Evaluate - conduct formative and summative 
evaluation of the instruction. 



Dick’s Instructional System Design Model 

Dick’s model (Dick, Carey, & Carey, 2001) is a 
linear model. It encourages a step-by-step process 
in which instructors are expected to finish 
processes in prescribed sequence. The output of 
one step becomes the input for next. This system 
design model consists of ten steps: 

1 . Assess need to identify goals. 

2. Conduct instructional analysis. 

3. Analyze learners and contexts. 

4. Write performance objectives. 

5. Develop assessment instruments. 

6. Develop instructional strategy. 

7. Develop and select instructional materials. 

8. Design and conduct formative evaluation of 
instruction. 

9. Revise instruction. 

10. Design and conduct summative evaluation. 



Figure 1: Two Established Instructional Design Models 



The second challenge faced by the ADLC team is the location and availability of our online faculty. 
Because of the Webster’s multi-campus structure, faculty teaching in the online programs are located all 
over the U.S. In addition, many of the online instructors are adjunct instructors. They have full-time jobs 
that often preclude them from working on course development during regular business hours, and are 
most likely teaching traditional evening courses at one of the University’s remote sites. This means that 
while they frequently need intensive support, faculty are rarely available to meet with our staff face-to- 
face. 



The ADLC Design and Development Model 

To meet these challenges, the ADLC team has developed a process that starts with a needs assessment, 
takes the faculty through a step-by-step design process, and then supports the faculty on an ongoing basis 
to ensure success in the online classroom. 
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General Approach 



The course development process begins with a needs assessment form that is filled out by new online 
instructors. This document is reviewed by the entire ADLC team and helps them assess the level of 
assistance the faculty may need and what particular challenges he or she might have. After the needs 
assessment, the instructional designer develops a suggested timeline in which the course will be 
developed, including target dates for each of the seven steps in the development process. At the same 
time, the course developer creates a new WebCT course shell based on the program template. 

At this stage, several training options are considered as well. On several occasions, groups of faculty 
have been brought to St. Louis, usually for 1-1 */2 days on a weekend, for intensive technical training. The 
structure of the training may change depending on the specific program. On one occasion, the course 
trainer traveled to a remote site to train a group of instructors at that location. Instructors are also 
provided with a variety of training materials and a WebCT Handbook. The course trainer is also available 
to work with faculty one-on-one in person or via phone/email, at any time during and after the course 
design process. Depending on timing and instructor availability, instructors may be at varying stages in 
the course development process when they attend training. The course design model, though, is designed 
in such as way that lack of technical knowledge in how to use WebCT will not stop faculty from starting 
the design process. 



Once the faculty member agrees to the dates and deadlines proposed in the course development timeline, 
the development process begins. At the start of each new step, the instructional designer emails detailed 
instructions and examples to the instructor and then maintains weekly (sometime more often) 
communication with them to offer assistance and encouragement. In addition to documents for each one 
of the steps, the instructional designer also provides written guidelines for developing content files 
(HTML pages), managing an asynchronous discussion, and more. 



The entire process is designed to be completed in a 15-week time frame that includes the initial needs 
assessments and final course sign-off. By providing a specific time frame for each step, as well as the 
design and development process as a whole, we help faculty manage their time and complete the course in 
a timely manner. 

The Design and Development Model 



To meet our needs, we have developed a new instructional design model that takes advantage of the 
ADDIE model and Dick’s ISD model, and adapt them to the online learning environment. Our model is a 
step-by-step process that leads instructors though the design process. This strategy allows for rapid 
development of online courses and easy project management. The ADLC model also encourages 
instructors to integrate WebCT tools into their course design in an effective way. The strategy involves 7 
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Generally, instructors are encouraged to go through these steps in sequence. For example, they should 
finish step 4 (Design a lesson module) before they begin Step 5 (Develop weekly content files). However, 
the strategy is open for instructors to reflectively design their courses. They can always go back and forth 
between the steps to refine their course design. 

Step 1 - Set instructional goals and select assessment methods. In this first step, instructors work with 
the instructional designer to write a syllabus or adapt their existing traditional course syllabus to the 
online learning environment. Course objectives, student expectations, course requirements, entry 
competency, course assessments, and grading information are discussed and reviewed. A page to 
introduce the instructor to the class is written in this step and a “getting started” activity is designed as the 
first class assignment to help students get acquainted with each other. In Step 1, faculty are also 
introduced to our Page Design Guidelines to help them create HTML documents appropriate for WebCT. 

Step 2 - Analyze content and buUd a course content outline. In this step, instructors analyze their 
course content and generate a detailed outline. The content outline should be aligned with the course 
objectives identified in Step 1 . Instructors work with instructional designer to lay out an instructional 
plan for the course, including the scope and sequence of course content, instructional activities, and 
learning assessments. Student learning styles and motivation issues are taken into account as well. This 
plan may be refined when instructors are designing weekly content later in Step 5. Upon the completion 
of this step, instructors create a word document that records the content outline and email it to 
instructional designer. 

Step 3 - Customize WebCT template. Each of Webster’s online programs has its own WebCT template 
to ensure that all courses in the program have a similar “look and feel.” Step 3 allows instructors to 
customize the program template further to meet their specific teaching style and content needs. The 
instructional designer works with instructors to identify and integrate the appropriate WebCT tools (e.g. 
chat room, discussions, quizzes, and glossary, etc) into the course design. The course developer makes 
the actual changes in WebCT. Step 3 also helps instructors conceptualize and visualize how the content 
will be delivered and how the instructional activities will be organized and implemented in WebCT. 

Step 4 - Develop a lesson module. In this step, instructors generate a lesson module for week 1. The 
module may differ by content and discipline, but usually includes lecture note pages, discussion activities, 
and assignment components. It may also include other components such as PowerPoint presentations. 
Instructors send these materials to the instructional designer for review. The instructional designer and 
course developer then work together to take these materials, upload them to WebCT, and make a 
prototype lesson module. Instructors can review the lesson module at any time and changes are made as 
needed. When all parties agree on the design and structure of the first lesson module, it becomes an 
example for the rest of course. 

Step 5 - Develop weekly content files and course components. Next, instructors are ready to develop 
all the remaining content files and course components. At this point, instructors can usually work 
independently to develop files for each week, following the content outline developed in Step 2 and the 
lesson module developed in Step 4. In Step 5, the instructional designer may be involved if instructors 
need assistance in designing or revising instructional activities. The course developer may also be 
involved to assist in converting files to HTML format or developing PowerPoint presentations. Once 
instructors finish the files for one week, they send them to the course developer to be loaded into WebCT. 

Step 6 - Build course on WebCT. In this step, the course developer uploads the instructors’ files into 
WebCT and builds the course site following the example of first lesson module. Step 5 and Step 6 can be 
completed in parallel; going back and forth until all files are built into WebCT. 
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Step 7 — Test and evaluate the course site. Once the course is built, the ADLC staff review the course 
to ensure that all links are working and all contend is in place. Course instructors receive a checkout list 
to help them with the final review of the course. Instructors go through all items on the list to make their 
courses are ready for the semester. At the end of the first semester teaching, the instructional designer will 
do a follow-up to get feedback from instructors about their experience teaching the course. 

Using the ADLC Model: Success and Lessons Learned 



Success 

Our strategy has been applied successfully so far. Since Fall 2002, we have used this strategy to develop 
1 Online MBA and 5 Online Security Management courses. We are currently in the process of 
developing 4 additional Online SECR courses and 1 Online MAT course. Of the courses already 
developed or currently in development, all but two of the instructors live outside the St. Louis 
metropolitan area, including Maryland, New Mexico, North Carolina, Virginia, and other parts of 
Missouri. All completed courses were built within the 1 5-week time frame and instructors are satisfied 
with their final product. In our follow-up email communication with instructors, they are doing well with 
their online courses. One instructor said that his online courses are “great” and his online teaching 
“successful.” 

Lessons Learned 

From our experience, we have found that the entire ADLC team, especially the instructional designer and 
course developer, must work together and be flexible in order to assist instructors in designing their 
courses. Each instructor has different needs, teaching styles, technical abilities, and experience, and our 
team has to accommodate these individual needs. Not all faculty will follow the design model step by 
step. Some instructors, for example, continue to refine their design as th^ proceed, they may come back 
to Step 3 to add/remove a certain design when they are developing step 5. 

In addition, we found that communication is crucial for the success of this strategy. The instructional 
designer takes on the responsibility for initiating and maintaining communication with the instructors 
throughout the process, and beyond. The designer and course developer also need to communicate with 
each other frequently while building an online course to coordinate their work effectively. 
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Abstract 

This study presents a brief summarization of the history of online high schools as a context for 
investigating the kinds of instructional methods being used in online high school courses. The framework 
for the investigation is typology of instructional methods, which is used to compare the methods utilized 
by three online high school course providers. The results of the study are presented with comments about 
the findings and suggestions for hiture research. 

Purpose of the Study 

This study focuses on instructional methods^ used in online high school courses. Instructional methods 
involve relationships between materials/resources, instructors, and learners. We can gain several insights 
from investigating the methods used in online high school courses: 

❖ Insights about commonly chosen instructional methods. 

❖ Insights about the kinds of higher order thinking skills that are encouraged. 

❖ Insights into the degree to which the strengths of computer technologies are being used to 
strengthen and support instructional methods. 

Methodology 

A convenience sample of courses was chosen from three institutions'* which provide online high school 
courses (Table 1). Our intent was to observe the types of instructional methods in each of the courses. 
Course Providers and Courses Reviewed 



^ Instructional methods are defined as “ways of helping someone to learn” (Reigeluth, 2001). In order to have a 
standard for comparing online methods we chose a typology of instructional methods developed by Dr. Michael 
Molenda at Indiana University (2001). The categories in this typology are based primarily on the essential 
communication pattern entailed in each particular method; that is, the relationships between Teacher (or teaching 
system). Learner, and Resources. 

^ Apexleaming.com is a for-profit company, which fills the advanced placement (AP) niche of the online high 
school market. Class.com is a for-profit company that is based in Lincoln, Nebraska enrolling students in all 50 
states and serving the international market. Cyberschool courses, unlike those from Class and Apex, are publicly 
accessible 



ERIC 



170 



Keller & Cakir ❖ 171 



Table 1. 




Literature and 


• Algebra 1 


• American History 


Composition 


• Civics 


• Classic Literature in the 


Physics 


• U.S. History 


Western World 


Statistics 


• World Literature 


• Economics 


U.S. History 




• General Biology 


U.S. Government and 




• Statistics 


Politics 







Prior to examining the online courses, a one-hour time limit was established for viewing each course. 
Both researchers reviewed one of the courses in an effort to establish uniformity in the reviewing process. 
Each researcher used a data collection instrument to record instances of instructional methods in each 
course. We were most interested in the variety of methods employed rather than the frequency of their 
use. 

Results 



Apex 

Apex courses have a consistent pattern in terms of instructional methods (Table 2). The courses have a 
standard look and feel. The syllabus page consists of hyperlinks to course content, assignments, and 
activities. The syllabus hyperlinks have titles reflecting Ae instructional methods employed in each 
section of the lesson. These instructional methods do not necessarily fit the categories outlined in the 
typology. For example, the tutorial sections of the Apex lessons fit the description of presentation in 
Molenda’s typology. The purpose of these courses is to prepare students for the Advanced Placement test. 
The true success of students in these courses is measured by a standardized assessment, the Advanced 
Placement exam. 



Table 2. 

Observation of Instructional Methods in Courses. 



Cyberschool 






5>codrses ’ ■ 



'Total; 



Insiiiictibnd . Apex , • Class 

Presentation 5 4 5 14 

Drill 5 2 18 

Reflection 1 2 4 7 

Game 0 Oil 

'-Eabdf^^ 
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Class.com 



Unlike Apex, Class courses (Table 2) do not follow a template and so do not have a standard look and 
feel. The most employed instructional methods in Class courses, according to Molenda’s typology, are 
presentation, reading, and discussion. The presentation of content includes many multimedia items. 

Cyberschool Courses 

Cyberschool courses (Table 2) vary in their look and feel, as do the assignments and the instructional 
methods employed. Despite a lack of stylistic conformity. Cyberschool courses employed the most 
varied instructional methods among all of the courses we reviewed. Laboratory work included 
simulations, an outside activity, and collaborative partner work. All methods from the typology were 
observed in Cyberschool courses. 



Discussion and Interpretation of Results 

Without making any generali 2 ations, we would like to point out a few observations and offer a few 
reflections based on our examination of the methods used in online high school courses. 

❖ Presentation plays a key role in almost all courses observed. Teacher centered instruction 
continues to dominate in the online classroom. 

❖ Reading enjoys a prominent position in the online course. Copies of readings are available 
on line. Textbooks continue to be employed. Hyperlinks provide primary and augmenting 
content for online courses. 

♦> Few courses capitalized on the strengths of the computer to connect individuals 
synchronously and asynchronously. Only one organization offered courses with 
collaborative work on assignments. 

❖ Few courses implemented reflection, games, or laboratories as instructional methods. 

❖ Few courses used the drill or tutorial methods as defined by the typology. 

What are the reasons for the observations above? Any answer would be incomplete but here are our 
reflections: 



❖ Instructional methods are not always chosen for their pedagogical appropriateness. Ideally 
instructional methods should be thoughtfully matched to objectives and desired learning 
outcomes, this is not always the case. The high cost of developing online courses, the 
convenience of transforming the face-to-face materials to online materials, and the high 
cost of supporting the delivery of online courses can be contributors to poor matches 
between optimal instructional methods and the course objectives. 

❖ Some instructional methods are easier to use than others. Presentations are comparatively 
easy to prepare, implement, and administer in comparison to laboratories. While 
laboratories may be well matched to instructional content and desired learning outcomes, 
they may not be leveraged because of the extra work that they entail. 
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❖ Exploiting the strengths of the computer medium is a skill that many educators and course 
designers have not mastered. Educators have largely designed for print-based media in the past. 
The strengths of the computer media such as, response tracking, instant feedback, calculating, 
sorting, and archiving are still being discovered and applied gradually. Mapping the strengths 
of the traditional classroom and print-based venues onto an online course invites less than 
satisfactory results. 

❖ There is no surprise that presentation and reading were prominent features of all online courses 
that we reviewed. These instructional methods have been used throughout the history of 
instruction and continue to be appropriate methods in the online delivery medium considering 
their text-based orientation. The methods that are little used such as games and laboratories 
deserve a closer look because of their capabilities to support learners in active meaning making. 

Conclusions and Suggestions 

Moore (1994) says that successful secondary teachers employ a variety of instructional methods. This 
indicates that successful course design would utilize a variety of instructional methods. The results of our 
study point toward a need for a greater variety of methods to be implemented in online courses. Online 
courses can be improved by using a variety of appropriate instructional methods to address learning 
needs, content, and desired outcomes in varied learning situations. 

According to Merrill’s First Principles of Instruction (2001), good instruction starts with a problem and 
includes activation, demonstration, application and integration. Merrill hypothesizes that failure to model 
the First Principles will result in less effective instruction. The high school courses, which we reviewed, 
did not reflect problem solving as the central focus of the course design as is evident by the few instances 
of the laboratory method in our data. 

Further research about online high school courses could pursue a number of paths. Investigation of a 
broader range of courses from a greater variety of companies could yield a more complete picture of 
instructional methods being employed. Comparisons of student achievement across courses employing a 
variety of instructional methods could provide insight into the methods most effective. Probes into 
successful implementation of laboratories, games, discussion, and reflection could provide guidelines for 
course developers desiring to leverage these high-powered methods in their courses. 

The anytime/anywhere promise of online courses brings an important consideration to course design. 

More people are going to be exposed to a single course design than before. It is very important that these 
widely distributed designs are of high quality. This quality can be measured in part by the match of 
instructional objectives and content to the instructional methods used, and the degree to which the 
strengths of the medium are exploited to accomplish tasks and to support learning in ways not possible 
without the computer. 
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Demonstrating the Scholarship of Teaching Online 
Using MERLOT 
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In recent years the increased attention is being given to teaching activities as part of the promotion and 
tenure guidelines even at the Doctoral/Research Universities — Extensive Universities (formerly Carnegie 
Research I category). Our own university is in the midst of rewriting its policies to make teaching 
activities even more inclusive in the promotion process. My personal experiences are as Faculty Senate 
President and former chair of the Chemistry Department as well as a member of the MERLOT Co- 
Leaders team. 

However, there is a also a very sizeable group of faculty who express concerns that the new emphasis on 
teaching is, in some way, diluting the role of research and allowing routine teaching to over counted. 

Those faculty members have a legitimate concern and one way to prevent the misapplication of the new 
guidelines is by better-defined criteria for excellence in teaching. 

Ernest Boyer and others have argued that to really advance the status of teaching requires us to develop a 
scholarship of teaching. There is now an extensive Carnegie Academy for the Scholarship of Teaching 
and Learning, CASTL (http://camegiefoundation.org/CASTL/index.htm) as well as many other groups 
such as those at the American Association for Higher Education and the Center for New Designs in 
Learning and Scholarship, CNDLS, at Georgetown (http://candles.georgetown.edu/index.htm). In order 
to develop instmction as scholarship we need to apply review criteria to teaching activities of similar 
exactness to those we use in evaluating our research activities. One reason of the many reasons teaching 
has been so underemphasized is that quality of teaching is so hard to measure. Most of you have been 
involved in promotion and tenure decisions where the quality of teaching must be evaluated. At LSU we 
routinely, and have for about 20 years, include the results of student evaluations of teaching. Our 
experiences suggest that these have some use and do define the very best and the very worst but in their 
simplest application do not provide administrators or other faculty much guidance for judging the vast 
majority of our faculty. In fact, Charles Dziuban of the Center for Research Initiative in Teaching 
Effectiveness (RITE) at the University of Central Florida, in a very extensive study does find some 
validity in those ratings. They also found that the entire multiple-choice survey could be done with one 
question, all questions are strongly correlated; at least that is how the students use them 
(http://pegasus.cc.ucf edu/~rite/index.html). Many schools are now using a multitude of measures to try 
to get a better picture of a person's teaching accomplishments but still that is largely based on their 
classroom behaviors as measured against local standards. 

When we judge a person’s research, which most groups tend to say is easier than judging teaching, what 
we do is something rather different. Sure we judge how that person presents seminars and interacts with 
her/his peers but the real value of a person is what new discoveries that person has made and how that 
person has promoted or published that work. In research, if a person does not publish the work it is 
useless to society and we do not reward that faculty member. Furthermore, that research will not be 
published unless it meets the standards of the profession as judged by his/her peers. 

To judge the quality of instructional materials scholarship, especially when those materials are used in 
online instruction, we need to be able to judge the scholarship of the teaching as well as we now 
demonstrate the research activities of our faculty for promotion and tenure decisions. To do this we need 
a new set of tools and some collectively agreed upon standards of quality. We now consider one aspect of 
this problem, namely standards and a mechanism for evaluating the quality of online materials. 
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One aspect that is often downplayed is the amount of work necessary to develop quality materials. That 
time represents a major commitment of the part of the faculty to the profession and as such needs to be 
shared extensively. Also since the development requires so much work, one needs to share the workload 
and make the results public, much as we do routinely with research results. But there are real problems in 
Just sharing one^s work. Granted there are some discipline specific databases and web sites as well as 
some commercial sites that are appropriate for certain disciplines and in some cases, even well advertised, 
but in most cases the faculty member’s work is sought out by word of mouth or a lucky hit from a search 
engine. Too often the only choice is for the author to give out his material with little recognition in return. 
In addition while most faculty want their courses to be individualized by being able to pick and choose 
what items to include, commercial sites, at least most of those currently available, want the user to buy all 
of the content or none — and at a price. 

According to Shulman (1998), “A scholarship of teaching will entail a public account of some or all of 
the full act of teaching — ^vision, design, enactment, outcomes, and analysis-in a manner susceptible to 
critical review by the teacher’s professional peers and amenable to productive employment in future work 
by members of that same community.” Notice the key words of public, review, analysis, and community. 

One organization that has begun to deal with this issue in a comprehensive way is MERLOT 
(http://www.merlot.org). The MERLOT organization is a collection of 22 major educational 
organizations, many being state Boards of Higher Education, which provides a location for the faculty to 
display its work in instructional technology. It provides a faculty driven database of online educational 
applets for anyone to use freely (in all senses of the word). It is an attempt to increase the dissemination 
of learning objects for faculty across many disciplines, improving the instruction for all while developing 
a real community of online material producers and users, while at the same time recognizing the nature 
and proper authorship of the material. Thus MERLOT is a large collection of links to online learning 
objects in 14 disciplines (6894 items with 1422 user comments and 463 reviews as of June 7, 2002). 
However MERLOT is different from most collections. These items are reviewed and commented on by 
experts and users of the materials. The reviewers also use well-defined criteria. So what is not fully 
appreciated by many is that MERLOT is above all a process as much or more so than a searchable 
database of quality materials. It is that process that is most important to us in the present context. 

Let us discuss this process at some length. MERLOT has to be understood at several levels, all of which 
relate to the scholarship of teaching. There are users, authors, members and nonmembers, reviewers and 
the editorial boards, all interacting communities. We start with a small educational applet URL submitted 
to site. It can be submitted by anyone: member, nonmember, or author. It is automatically indexed, the 
author is notified, and the item is made available to everyone via a very extensive search engine. 

Three things happen next. First, users can begin using these applets by going to the author’s web site. 
Those users are also invited to join the MERLOT community (it’s free) and submit comments on the use 
of this item. Secondly, when the review committee (or two people specifically in the case of chemistry) 
has time they will first triage the item based on their quick review and initial guess as to quality and 
importance (ranking them from review immediately to remove on a five-point scale). Thirdly, the 
editorial board assigns or the reviewers individually decide what items to review. Those reviews are 
performed according to published standards agreed upon within the discipline and consistent with the 
MERLOT general criteria. It is veiy important that there be such discipline specific guidelines. 

As an example, the chemistry discipline team drafted its criteria and review process after a very extensive 
period of discussion. There are three general categories of evaluation standards within MERLOT- 
Chemistry: 
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1 . Quality of Content. 

2. Effectiveness as a Teaching-Learning Tool. 

3. Ease of Use. 

These three categories are interdependent to some degree, but separate ratings are given for each. Specific 
evaluation criteria are applied within each category (see details at web site: 

http://taste.merlot.org/disciplines/eval_criteria/chemcriteria.html). Some criteria are applicable to several 
categories. While we will not cover all of the details here, it is important to note that the reviewers pay a 
lot of attention to the learning outcomes and good pedagogy in their reviews. We believe that it is the 
duty of our reviewers to make sure that the items are rated on their use and interactivity, and their learning 
potential even more than on the content and the ease of use. Most of those details are not published but 
form the basis of the two reviewers' final rating. The judgments of the reviewers, especially on 
effectiveness, are also different depending on the intended use of the materials as indicated by the author 
(e.g., Lecture/Demo, Tutorial, Homework, Lab - group or individual learning or other use). 

The authors are fully informed that a review is underway and at the end of the review process they are 
notified, just as in the case of a peer reviewed article in any respected journal. There develops an ongoing 
dialog with the authors about the review. The authors can make further comments to the reviewers just as 
in the case of a journal article. The reviews are briefly summarized by a star system (5 stars is 
exceptionally good) but more important are the details in the written comments. Those reviews are posted 
for all to see. One of the many other advantages of our reviews is that they alert the user to any special 
issues involved in their use: maybe a plugin is needed, maybe it works only with certain browsers, maybe 
there some obvious improvements that could be made, etc. We also encourage the authors or any 
member user to add online learning assignments for each item so that others may see how these items are 
used in some detail. In some cases entire midterm exams or homework worksheets are available based on 
some item in the collection. It is a very public process. 

At the end of the process, authors with items rated as high quality items have the option of having 
MERLOT inform their superiors of these reviews and in many cases these have played major roles in the 
promotion of faculty members. So MERLOT is not only of interest to the users (faculty and students) but 
to administrators who see the value of its reviews as illustrating the quality of the author's activities and 
thus his scholarship in educational activities. Since it is the collective view of many experts and users in 
the discipline, it has much more validity than other measures often used. These are also external to the 
university, which gives it national and even international validity. 

So what is MERLOT accomplishing by its actions? First of all, it is making the instructional process 
more public. It is developing a community of instructors dedicated to the scholarship of teaching, 
especially those involving technological applications. It is generating a set of defensible discipline 
specific standards by which items are judged by one’s peers. In short, it is trying to model the peer 
review process so familiar to us in evaluating research activities. 

Now clearly, MERLOT or any similar database cannot judge the complete merits of any instructor nor 
even begin to address how well the learning process occurs in each classroom but it does judge the merits 
of one of the products of that educational activity. 

We also suggest that the well-defined discipline specific peer review process by external experts used by 
MERLOT could also be adapted to one's own institution and to other collections. While the final reviews 
which MERLOT posts provide strong supporting evidence in tenure and promotion decisions and for 
various teaching awards, the process is even more important. It is hoped that institutions and individuals 
will join with MERLOT and adopt similar standards and processes that can be used in tenure and 
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promotions considerations. Some have even suggested that MERLOT or another agency could provide 
teams that could review instructional materials for promotion considerations. 

Have we made progress? I believe so. We are getting many testimonials from faculty members who use 
the MERLOT reviews in their P&T packages (see also “Ever So Slowly, Colleges Start to Count Work 
With Technology in Tenure Decisions,” Feb 22, 2002, Chronicle of Higher Education article by Jay 
Young or the Chronicle chat session with Bruce Mason from MERLOT and others in a live colloquy 
recorded at http://chronicle.eom/colloquylive/2002/02/tenure/). Many like the fact that their work is 
being shared and also that they can use the products of others and not spend time doing everything 
themselves, the “Lone Ranger” approach. The anonymous reviews by experts are highly rated in many 
packages. However, the bottom line is that ultimately a tenure decision is made by a group of faculty 
sitting in a small room. The administration tries to make sure those very private decisions conform to 
their university policies, which increasingly support the use of technology and newer teaching methods as 
well as good instructional designs as an integral part of faculty member’s promotional requirements. In 
my own experience I can see this happening and the use of peer reviews for both research and instruction 
(and instructional materials) is quoted widely in promotion materials today. It was not just a short time 
ago. I believe MERLOT is on the right track and setting the standards for others to follow. 
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Friedman and Fisher's (1998) Handbook on Effective Instructional Strategies: Evidence for Decision- 
Making made 1 4 evidenced-based teaching generalizations, built on research in face-to-face settings. This 
paper looks at “how” these 14 instructional strategies are incorporated into distance learning 
environments by using benchmarks explored by Friedman and Fisher. 

Instructional strategies are those tactics used by the instructor to promote learning. Friedman and Fisher’s 
classroom based findings support the effectiveness of the identified instructional strategies in improving 
student learning. The 14 strategies are generalizable across “academic disciplines, types of students, and 
educational settings” (xvii). 

Literature reviews (Abrami, 2001; King and Weber, 2001) demonstrated many instructional styles or 
strategies used in the online environment. However, few studies examine the micro level of instructional 
strategies as defined by Friedman and Fisher and fewer still that compare their effectiveness when used in 
the distance classroom, especially the online environment. An observational study was conducted to 
analyze whether effective instructional strategies found in the face-to-face environment could be 
identified in the distance learning environment. This study was conducted over two years with data 
extracted from observations recorded in student papers, coded and aggregated to identify if Friedman and 
Fisher's strategies could be found and what variations of the strategies might exist. For this paper, we are 
reporting instructional strategies in the online environment. They are shown in Table 1 with specific 
tactics and examples following each instructional strategy noted in the text below. 



Table 1. Strategy Categories and 14 Instructional Strategies from MI. Friedman and S.P. Fisher (1998) . 



Process strategies to enhance 
students’ learning and 
adaptation. Providing: 


Reminders 

Transfer of Learning Instructions 
Teamwork Instructions 


Teaching strategies to 
increase students’ learning. 
Increase students’ learning 
by: 


Taking student readiness into account 
Defining instructional expectations 
Providing effective evaluation and remediation 
Providing contiguity 
Utilizing repetition effectively 
Clarifying communication 
Reducing student/teacher ratio 
Providing subject matter unifiers 


Strategies to manage the 
classroom by use of 
appropriate time dimension: 


Providing ample learning time 
Keeping students on task 
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(Note: some of the tactics and examples below have been condensed and consolidated for space reasons.) 

Providing Reminders 5'/ro/egy-“Achievement of learning objectives is enhanced when reminders are used 
to cue the recall of information needed to perform assigned tasks” (p. 294). In the online setting: 

❖ Mnemonic devices used; highlighted words represented terms in glossary or were links to maps 

or examples. Examples used that illustrated application of learning to the real world. 

❖ Weekly audiovisual updates with PowerPoint and e-mail. 

Providing Transfer of Learning Instruction “Achievement of learning objectives is enhanced when 
students are taught beforehand the knowledge and skills needed to perform assigned tasks and how to 
determine when learned knowledge and skills can be used to perform . . .tasks” (p. 295). In online 
settings: 

*V Assignments given in “building block” format. Modules are related to previous modules and 
drawn from previous knowledge. Directed to additional or previous material to problem solve. 

❖ Students describe in discussion threads how current learning is relevant in their own lives. 

❖ Project sheets direct students to learn an objective, and then apply to real world situations. 

Providing Teamwork Instruction Strategy- “Achievement of learning objectives is enhanced when 
students are taught to perform complementary tasks as a team in pursuit of the objective” (p. 295). In the 
online setting: 

❖ Encourage collaboration and discussion; refer to responses; synthesize responses; provide 
information; offer insight; provide additional input and provide opposite perspective from 
participants and ask them to think about and redirect response. 

❖ Threaded discussion: Continual collaborative feedback enhances active learning socially, not 
competitively; prevents isolation of student. Assign person to lead and to start activity. 

❖ Providing tools for groups to give and receive constructive feedback including instructions to do 
so and examples of good feedback. Include FAQ that gives students answers to what to do 
when the group isn’t working as a group. Small groups established with 5-10 students as virtual 
classes within the larger class for discussion and a hi^er level of participation. 

❖ Study groups formed in response to student panic after first unit exam (statistics). 

❖ Groups of 4; points given for introductions in group room; grades for chapters are given as 
groups not individually; brainstorming within group -one member reports back for group. 

Taking Student Readiness into Account- “Achievement of learning objectives is enhanced when students 
possess the readiness capabilities to achieve learning objectives” (p. 293). In an online setting: 

❖ Course orientation quiz covering course requirements. First lesson reviewed basic concepts. 
Students are introduced first to concepts of distance learning as a prelude to course content. 

❖ From welcome screen students move to syllabus. The acceptance of the syllabus document is an 
assigned requirement, which confirms students understand expectations. 

❖ Students wrote a personal history paper that was content related at very beginning of class. 

❖ Division of material into units with each providing prerequisite skills for subsequent skills. 

*1* Program has step-by-step approach moving from simplest concepts to most 

complicated. 

Defining Instructional Expectation: “Achievement of learning objectives is enhanced when prior 
to instruction (1) learning objectives are derived for students, (2) procedures to be used in the 
performance of tasks to achieve the objectives are identified, and (3) student outcomes 
designating achievement of the objectives are defined” (p. 293). In an online environment: 
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❖ Initial face-to-face or videoconference meeting. Expectations are restated in e-mail to the 
student. 

♦> Syllabus provided grading scale, performance expectations, course outline, and focused toward 
learning objectives. Define objectives based on ABCD=audience (the students), behavior 
(learning task/performance), condition (how to learn the objectives), and degree (performance 
criteria). Detailed explanation of how web course will work. 

♦> Percentages assigned to group participation, individual participation, and projects helped to 
define what was required to achieve course goals. Performance criteria are matched with 
checklists provided throughout the course to ensure students were meeting expectations. 

❖ Getting-started section to articulate structure of course. Examples of work/lessons as guideline. 

♦> Classroom profile section provides listing of completed activities. Step-by-step instructions 

specifying what procedures students were to follow within the course. 

❖ Discussion is considered a key course requirement; requirement for 350-500 word essay 
response to instructor’s questions. 

❖ Definition from course catalog and then teacher’s definition of course provided. . .using o\vn 
words. Prerequisites listed to give student a “sense” of being qualified. 

Providing Effective Evaluation and Remediation: “Achievement of learning objectives is enhanced when 
appropriate remediation is provided: ( 1 ) evaluation procedures and remedial tasks are formulated when 
task sequences are planned, (2) student task performance is frequently evaluated, (3) feedback on 
evaluation is given to students without delay, and (4) incorrect performance is immediately remediated, 
based on evaluation results” (p. 293). In an online environment: 

❖ Grade book is included that may be accessed at anytime. 

<♦ Pre-test, post-test, mid-quarter, and final exam testing on objectives to ensure students learn 
what they need to learn. Feedback given as students perform exercises and practice sets. Self- 
tests with immediate feedback. Unit projects email to instructor with 1 -3 day turn around for 
grading. 

❖ Continual feedback throughout the course reinforces expectations. 

❖ Teacher marks student responses and student redirected if material not understood. Students 
allowed to redo notebook and unit projects to improve grades. 

❖ Positive feedback on well-done assignments and suggestions for improvement on others. 

❖ After each lesson give quiz of 5-7 questions. May retake quizzes. Immediate feedback with 
comments on incorrect answers. Students may use threaded discussions to discuss quizzes. 

♦J* Contact instructor-email provided; also participates in threaded discussions. 

Providing Contiguity: “Achievement of learning objectives is enhanced when events students are to 
associate are presented to them close together in time and space” (p. 294). In the online environment: 

❖ Posting assignments for other students to comment on before submitting to instructor; 
substantive-useful feedback from instructor before next learning objective. 

❖ Material is organized according to subject-matter topic and assignments are grouped by subject- 
matter topic. Course material is structured so that the assignments build upon each other. 

❖ Lessons presented sequentially utilizing information from previous lessons. 

❖ Assignments divided into small manageable amounts of material with concepts building on each 
other; students began by reading or researching about concept followed by activity or 
assignment. 

Utilizing Repetition Effectively: “Achievement of learning objectives is enhanced when there is repetition 
in instruction and tasks students are assigned to perform” (p. 294). In the online setting: 

❖ Vary assignments within the same topic to enhance deep learning. No single explanation is 
clear to all students. Final exams are an accumulation of quiz and test questions. 
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❖ Students discussing learning of activities through discussion threads (even though some of the 
comments may be repetition) helps to provide repetition of information for deeper learning. 

❖ Important information repeated several times and in several locations. 

❖ Varying activities to reinforce learning followed reading; graphics and pictures repeat concepts. 

❖ Test questions on final were from earlier assignments. 

❖ Units are broken down into lessons. Summary ties all together at end of unit. 

Clarifying Co»jOTMn/cat/o«-“ Achievement of learning objectives is enhanced when information on 
objectives, tasks, and evaluations is clearly communicated to students” (p. 294). In the online setting: 

❖ Constant direction to resources. Information provided for technical support. 

❖ Multimedia used provides stimulation to participation and engage different learning styles. 

❖ Syllabus-succinct and directive; expectations-reading studying and writing. 

❖ Format of class including quiz schedule, projects, attendance policy, supplemental material to 
lecture, video presentations, access to college library, requirements for communications (email 
and website discussion board); office hours and encouragement to communicate. 

❖ Thorough outline of class schedule; include student names that lead discussions. 

❖ Grouping tasks together for each assignment and specifying how each assignment related to 
different learning objectives. Familiar items selected to demonstrate concepts. 

♦> Clarified with e-mail, discussion area and on several web pages. 

❖ Videotaped lectures in adjunct to online course; weekly audiovisual updates with PowerPoint 
and e-mail. Instructor takes an active role in threaded discussions; frequent clarifying questions 
to eliminate confusion and encourage participation. 

❖ Course orientation quiz covering course requirements and stressing the contact information for 
the instructor. After quizzes instructor available by phone. 

❖ Study Guide defines procedures step by step and encourages students to contact instructor to 
verify understanding; information and project sheets are clearly written with precise 
instructions. 

❖ Re-clarifying assignments and providing resources within the course for additional clarification. 

❖ Journal writing was used to relate content to students’ experiences. Numerous graphics and 
links to illustrate concepts; observations, research, demonstration of concepts. 

Reducing Student/T eacher Ratio: “Achievement of learning objectives is enhanced when there is a lower 
student-to-teacher ratio in teaching situations” (p. 294). In the online environment: 

❖ Ratio for this online course is typically 11:1, however, this evaluation period was 4:1; and still 
team-taught. Automation of grading quizzes gives instructor more free time. 

❖ Interaction done individually without known numbers in class. Student has no knowledge of 
how many students in class. Ratio of 6:1 allowed for individuation of instruction. 

Providing Subject Matter Unifiers: “Achievement of learning objectives is enhanced when a scheme is 
used to highlight parts/whole relationships in the subject matter students is assigned to learn” (p. 294). In 
the online environment: 

❖ Models and diagrams are used and developed by students; problem based/knowledge based 
environment to create higher level of thinking. 

❖ Rich in research, design and format -related links, metaphors, recommended readings. 

Providing Ample Learning Time: “Achievement of learning objectives is enhanced when students are 
given ample learning time to perform tasks" (p. 295). In the online setting: 

❖ Optional extra credit quizzes are to be done during the assignment that is scheduled for that 
timeframe. Re-dos on papers were encouraged. 

❖ Course was extended to a 10-month period because it was a writing course. 

❖ Students are given one year from registration date to complete class. Self paced class. 
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❖ A major question is posted on one day of the week and must be answered by noon Mondays. 

❖ Assignments and tasks were posted at the beginning of the course, as were due dates. 

Keeping Students on Task: “Achievement of learning objectives is enhanced when students spend more 
time attending to tasks formulated to enable them to achieve the learning objective” (p. 295). In the 
online environment: 

❖ Clarifying expectations and timelines in advance help students focus; reminders; encouraging 
students work ahead. 

❖ Assignments were specifically related to the required reading and goals/outcomes of the course- 
no “busy work”. Weekly agenda with list of assignments, instructional activities, and 
evaluation; all information is in order and on the web. 

❖ Hyperlinks to additional resources so time is used efficiently; future lessons not available to 
prevent working ahead and distraction. 

❖ Tasks are performed in the field and then reported in to Web site. Teacher is not with them 
during field tasks. Tasks had to meet a timeline. 

Implications and Conclusions 

In reviewing over 50 online courses, we determined that all 14 instructional strategies were being used. 
We feel this reinforces that "good teaching is good teaching" and that online teachers exhibit the ability to 
use instructional strategies identified as successful in face-to-face teaching. We would also note that our 
online instructional strategy research appears to link to and reinforce other teaching and learning theories 
and applications (APA, 1997; Chickering, et al., 1987, 1991, 1996; Sattler-Weber, Hunt & King, 2002). 
We also note that three strategies identified here, “Defining Instructional Expectations,” “Clarifying 
Communication,” and “Evaluation and Remediation,” may be the most eflFective of the 14 strategies, or 
may be the most easily adapted to online teaching as evidenced by the number of strategies observed. 

Instructional strategies may be seen as analogous to what is found in the body that rejuvenates cellular 
activity, DNA, often referred to as the “essence of life.” Like DNA, instructional strategies are 
responsible for the enormous possibilities for variability at the micro level — the “essence of learning.” 
Used in the online environment, the instructional strategies are energizers of the learning experience 
while technology is the injection system or carrier. Ways in which specific instructional strategies are 
utilized and operationalized will differ when the characteristics and capacities of technology in the online 
environment are analyzed. Principles and practices are the invisible threads of interrelated actions and 
teaching theory that make up effective strategies. In building an instructional paradigm, teaching in the 
online environment must be both purposeful and mindful of the interrelatedness of principles and 
practices. While these 14 strategies are the “DNA” of good teaching, the applications are everyday 
activities most teachers commonly use. We believe that we can show that these strategies are effective in 
the distance education environment as well as the fece-to-face setting. 
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Online Education Investment Strategy: 

Key Educational and Financial Success Factors 

David G. Kraenzel, Ph.D. 

Continuing and Vocational Education (CAVE) 

University of Wisconsin-Madison 

Introduction 

The Chronicle of Higher Education, the Wall Street Journal, and the New York Times contain 
almost daily announcements of yet another university or business enterprise offering a degree via 
the World Wide Web (WWW). With almost the same frequency, these newspapers contain 
stories about online educational programs that have been deliriously successful or have failed 
miserably to achieve their educational or financial objectives. It would appear that many 
universities are investing in online education without full knowledge and understanding of what it 
takes to implement high quality online educational programs. Such “boom or bust” variance 
clearly demonstrates an escalating need for advanced research targeted at discovering key success 
factors (KSFs^) affecting online education program management. KSFs are the assets and skills 
that provide the bases for competing successfully. Examples such as state-of-the-art knowledge, 
technical or management skill acquisition and positive attitudes will surely help all of us improve 
resource allocation in higher educational investments that ultimately benefits our learners. 

It sounds like we need to consider an educational venture perspective in our strategic judgment. 
Hmmmmmm, that calls for a little bit of educational investment discussion. What is the reality 
here? Are we going to have to earn our way? Maybe to some extent since online education is 
still an innovation in education and any educational innovation is usually put to the “does it pay 
its own way test” on its journey towards a distributed learning portfolio. An educational 
investment may mean to spend (time, effort, money) with the expectation of some satisfaction 
(student, faculty or institutional) as opposed to putting only (money) into a business, real estate, 
stocks or bonds, for the purpose of obtaining an income or profit. So, where does the value of 
such educational investment research reside? 

The value resides in the comprehensive benchmark knowledge that contributes to the distance 
education field about online programming resource use and management. Educational KSFs such 
as faculty satisfaction, student satisfaction and learner support for example; and financial KSFs 
such as revenues, costs and ratio performance measures that have surfaced, serve to inform and 
answer the original research question asked in this inquiry: what is required to design, develop, 
implement and evaluate high quality online distance education degree programming investments? 

OK, so what are the KSFs? Well there are: I. Performance Measures; II. Revenues and Sources 
of Funding; ID. Costs and IV. Dominant Themes and Patterns. The performance measure KSFs 
are presented in two groups: educational and financial. Assuming we are educators at heart, we’ll 
look at the educational KSFs first. These fall into four major categories: learner-oriented; 
faculty-oriented; corporate/industry-oriented; and program-oriented. Each of these categories has 
numerous subcategories. Some are mentioned in this summary and a complete list of 
subcategories will be available upon request. 



’ Source: Aaker, David A. (1995). Strategic market management. (4* ed.). New York: John Wiley and 
Sons. 100-102. 
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Key Success Factors in Online Education: Performance Measures 

Educational Categories 

1 . Learner-oriented-, student/customer satisfaction; student performance; learning evaluations; 
interaction evaluations (learner-instructor, learner-learner, learner-content and learner- 
context); coursework review/maintenance/revision; quality (expert-opinion, face validity or 
cadre credentials, G.P.A. as a proxy, online program/course efficacy). 

2. Faculty-oriented: satisfaction; faculty involvement; and faculty/teaching improvement. 

3. Corporate/industry-oriented: satisfaction; involvement; and teaching involvement. 

4. Program-oriented: entry requirement reviews; self-evaluations; annual summary of program 
issues; annual statistical summary (applicants, admissions, enrollments, cohorts, 
conditionals, retention rates, graduation rates); annual percentage of tenure-track faculty 
teaching in the program; annual summary of student credentials; annual comparison with 
other programs; university measurement and evaluation survey; and post-program surveys at 
graduation and three-year intervals (graduate success rate in working world, employers, 
earnings levels and repeatability as students). 

Financial Categories 

1. Annual break-even analysis: with zero-sum requirement, without zero-sum requirement. 

2. Net cash flow management 

3. Cost/student/course 

4. Cost per program 

5. Net profit margin 

6. Pricing structure 



Revenues and Sources of Funding 
Revenue KSFs are presented in four major categories: 

1 . Online program development funding sources (state funds, federal programs and agencies, 
foundations, in-kind support and corporate/industry sponsorship). 

2. Annual tuition (program, instructional and in some cases segregated fees) for programs. 

3. Technology fees. 

4. Services and service agreements (research updates, mini-talk/seminars, direct billing, credit 
card fees, in-kind support and service agreements). 
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Costs 



The cost KSFs are presented in seven major categories. The very detailed nature of costs 

precludes listing all subcategories in this summary due to space restrictions. There will be 

handouts available during the presentation that will provide these details. 

1 . Marketing/market research/public relations/promotion: (pre-program needs assessment, 
marketing plan). 

2. Management (salaries, fringes). 

3. Program Development (annual loan, funding principal and interest repayment, opportunity 
cost, overheads, supplies and maintenance, course development) 

4. Program Delivery (Instructional costs, technology platform, and courseware) 

5. Learner Support (1-800 number for instructor/TA, 1-800 number for interactive conference 
bridge, technical, proctoring, library services, and advisor (counselor). 

6. Summative Evaluations 

Dominant Themes and Patterns 

1 . Universities were distinctively innovative in their educational approaches to online 
programming implementation. The characterization was predominately entrepreneurial, 
market-oriented, a business plan approach, although one University was more cost-oriented 
while maintaining a “business consciousness or awareness”. 

2. University units were determined to be risk-takers to varying extents. 

3. The primary market segment targeted was working alumni. Other segments were added as 
each University discovered additional educational needs. 

4 . Faculty involvement and related issues topped the list of online programming barriers. 
Related issues included: faculty burden when online courses were expected to be added to 
their existing teaching load; copyright issues; reluctance to change; and the invisibility of 
online work in the tenure/promotion process. The second barrier was adequate, timely and 
annual funding. Other barriers discovered were: University-related, i.e. lack of 
administration commitment, inappropriate/inadequate guidelines/regulations; a lack of 
understanding of the marketing and competitive issues involved in the educational 
marketplace; inadequate University accounting systems for distance education program 
management; a significant dilemma where, for example, a twelve month in-University 
program approval process ultimately endorses program quality but, severely hinders timely 
market entry; lack of distance education history and relative newness of the concepts 
compared to on campus teaching and learning; and legislative control of tuition. 

5. The over-riding theme concerning improvements in fmancial matters was stable funding (or 
removal of volatility) and financial accountability. 

6. Educational application was priority over infrastructure or administration applications. 

?• The use of government funding in the development or startup stage of online education to 

give institutions the risk-taking capability to “test the waters” before traditional funding 
sources were adjusted or developed to provide standard line items in the annual budgets. 
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8. Although a zero incorporation of funds was prevalent at the inception of most distance 
education programs, it is reasonable to deduce that the general trend was toward a growing 
inclusion of these funds into base-operating budgets over time. 

9. The majority of respondents felt senior management already does or should hold back a 
percentage of general operating funds before the Dean’s or heads of departments/units 
receive their share for operating. In one University operating under institution-wide 
centralized management, respondents strongly favored this approach for uniform quality 
control and organizational consistency. For example, historically, one Dean advocated the 
building of a particular online program. He/she was then replaced by one who didn’t. The 
program was dropped until a third Dean appeared, again an advocate, thus resulting in the 
entire program having to be rebuilt. 

Implications for Practice 

This research builds online distance education theory based on the foundational works of Bates 
(2001); Coldeway and Spencer (1982); Douglas and Harmening (1999); Ehrmann and Milam 
(1999); Jenny (1996); MacDonald and Gibson (1998); Markowitz (1987); Moore and Kearsley 
(1996); Morgan (2000); Murgatroyd and Woudstra (1989); Rumble (1988); Smith (1998); 
Woudstra and Powell (1989); Yin (1994); and others. The inquiry hopefully provides useful tools 
and a benchmarking database for practitioners. 

This study took advantage of wonderful opportunities presented by rapid technological and 
cultural changes taking place in education. The momentum of the competitive forces at work in 
the educational marketplace was harnessed for practitioners in online programming, and is just as 
helpful to those traditional practitioners who face similar and increasingly competitive forces. 

We as educators may see the integration of online distance education into a more “distributed” 
teaching and learning model. We may also see a strong teaching and learning model emerge 
from the ashes of the struggle between online distance education and traditional education. The 
result may be a teaching and learning model symbolizing the immortality of education, where 
each constituent educator can preserve their own identity, yet can draw on and contribute to the 
synergies gained from the similarities as well as the differences with one another. 

Implications for Further Research 

This research discovered key success factors in online programming by identifying revenue, cost 
and performance measures and definitions that contribute to a common language to inform higher 
education. Systematic inquiries exploring the relationships between these identified variables 
could further inform the field. For example, research directed at putting actual numbers into the 
categories. These comparisons would enhance critical funding priority questions that arise such 
as; should Learner Support Services or Faculty Training be increased to improve program 
quality? Or where is our best educational return on investment? 

Intangibles such as goodwill or public relations are not accounted for in the research results. On 
the other hand, the value of faculty involvement, goodwill and repeat customers for example, was 
a point made by most study respondents. Therefore, a gap in organizational returns or revenue 
exists. How are very evident intangible returns valued by the organization? Are there others? 

A study of risk-takers (i.e., “risk seeking” versus “risk-neutral” versus “risk adverse”) operating 
in the University context would be valuable. What are the various organizational contexts that 



ERIC 



190 ❖ Online Educatiai Investment Strategy 



188 



encourage or support creative online programming? Are risk takers rewarded by the 
organizational structure? If so, in what ways? If not, why not? 

What about removal of zero-sum accounting constraints? What is the probability that removal of 
zero-sum accounting constraints will positively impact the educational mission of the University? 
Will such a critical turn around be accepted? Is such a move financially feasible? What impact 
will it have on those units serving a social need not served by a profit motive? 

Epilogue 

Please recall the original research goal was to determine key success factors (KSFs) by 
identifying revenue, cost and performance measure categories, subcategories and definitions that 
need to be considered in designing, developing, implementing and evaluating online distance 
education program investments to build online distance education theory. The literature revealed 
contextual issues at work in the University environment lending insights into subjective 
influences affecting the success of such investments. Therefore, the refined research goal was to 
answer: What are the revenue, cost and performance measure categories, subcategories and 
intrinsic definitions that need to be considered in designing, developing, implementing and 
evaluating online distance education programs to build online distance education theory? What 
are the institutional contexts and program characteristics influencing those categories and 
definitions? The benchmark information presented was derived from selected online education 
programs at four public, Doctoral/Research-Extensive Universities having characteristics that 
suggested successful programming investment or potential. 
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Pedagogy, Multimedia, and Distance Education: 

Developing Critical Inquiry Through Virtual Communities 

Dr. Don H. Krug 
Associate Professor 
The Ohio State University 

Emerging technologies offer educational possibilities for enhancing the pedagogical development of 
distance education courses delivered through the Internet. In this paper, I will discuss some ways that 
integrated multimedia has been used to strengthen the teaching and learning of a virtual education course 
— “Ethnic Arts Online.” From the beginning development of the Ethnic Arts Online course, it was 
important that it meet the same high academic and pedagogical goals of the face-to-face course. Over the 
past few years, I have collected data from these courses and relationally analyzed them with regards to 
content robustness and academic rigor. Course data was available from both courses primarily through 
student writings and interactions. From my research and examination of electronic educational 
computing courses, 1 believe that virtual teaching and learning can be academically challenging and the 
content contextually rich. I support a social constructivist educational position guided by critical inquiry 
about the complexities of life-centered issues, exploration of meanings and values of social, historical, 
and cultural contextual conditions, and dialogue about multiple and divergent interpretations. Below 1 
briefly introduce the Ethnic Arts Online course and some pedagogical aspects of integrated multimedia, 
critical inquiry, intercultural communication spaces, and student self-directed learning. Woven within this 
review are suggestions for developing and practicing social constructivist educational pedagogy. 

Ethnic Arts Online Course 

“Ethnic Arts; A Means of Intercultural Communication,” (Ethnic Arts) is an undergraduate course 
offered at The Ohio State University (OSU). Six or more sections of the course are offered each quarter 
through the Department of Art Education. Ethnic Arts satisfies three OSU undergraduate General 
Education Curriculum requirements: arts and humanities, social and cultural diversity, and second level 
writing. Course content is thematically organized to examine a comprehensive range of artists, artwork, 
and critical writing in the art worlds from diverse ethnic cultures in North America. Ethnic Arts enrolls 
approximately 500 students each year from a broad spectrum of the university's student population (e.g., 
Attcan American, Asian American, Chicano/a, Latino/a, Puerto Rican, Korean, Malaysian, Chinese, 
Indian American [India], Somalian, Mexican American, Honduran, European American, Native 
American, etc.). One section of the distance education course (Ethnic Arts Online) has been offered over 
the Internet each year since the 2000 spring quarter. Ethnic Arts Online combines course management 
aspects of WebCT within an HTML set of frames, along with animation, and multimedia authoring 
features of Apple QuickTime, Flash, and RealPlayer. Students enrolled in this online version of the course 
meet only in a virtual community. 

A distinctive feature of the Ethnic Arts course is that students interview local artists from various ethnic 
groups and within the student’s local community. In the face-to-face course, many of the students bring 
objects, images, and video in to the classroom to share with their classmates. They document the artist’s 
sense of ethnicity and cultural identity in relationship with their art-making processes. So in order to 
provide a similar rich context for distance education students, it was desirable to have intercultural 
communication spaces available for students to post visual, audio, and textual information via the Internet. 

Social Constructivist Pedagogy 

The Ethnic Arts Online Web site’s interface was developed with a “pedagogical design” to enhance 
academic rigor and content robustness. The pedagogical design infused aspects of integrated multimedia. 
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critical inquiry, intercultural communication spaces, and self-directed learning that were strongly 
influenced by a social constructivist educational philosophy. The aim was to promote a sense of initiative, 
creativity, collaboration, and responsibility among students and the instructor. 

In “Building Learning Communities in Cyberspace: Effective Strategies for the Online Classroom,” 
Palloff and Pratt (1999) suggested that the importance of developing and maintaining valuable online 
learning begins with interactions among students themselves, between faculty and students, and with 
collaboration in learning that results from these interactions. In describing the evolution of a learner- 
centered virtual community, these authors suggest that instructors need to find ways to make students feel 
“embodied” with the course content. Indeed, I have found that one of the most significant ways of 
involving students with online course content is to establish minimal levels of participation. This means 
students are required to take responsibility for their own learning, conduct inquiry, and make use of the 
course’s intercultural communication spaces. The development of integrated multimedia that supports 
involvement and interaction is critical to achieving these educational goals of content robustness and 
academic rigor in an online course. 

Integrated Multimedia 

The Ethnic Arts Online course incorporates many different forms of emerging technologies including 
Internet and Web-based computer-mediated communication, WebCT course management software, 
streaming audio and video, and virtual reality QuickTime movies. Macromedia Flash was used to 
organize content, multimedia, and course management apphcations within a set of html fi-ames. In 
addition, the course interface was visually arranged to guide students in recognizing and reinforcing their 
critical inquiry processes during their virtual experiences. 

The use of emerging technologies in the Ethnic Arts Online course draws from various forms of 
integrated multimedia, while creating a high expectation level for learning, it also facilitates self-directed 
inquiry and provides an environment conducive to collaboration among learners and the instructor. These 
resources have the potential to enable students to participate in a learning environment, inquire and 
communicate about ideas and issues, and assess their learning performance. The following overviews 
some of these resources: 

1 . Course Readings: online articles about course ideas. 

2. Contexts: lectures, audio and video materials prepared for students to view online. Audio and video 
are streamed and can be accessed from this resources link in the weekly assignment section. 
RealPlayer Basic must be installed in order to access this material. 

3. Unit Galleries: examples of past student papers written about selected artists, art forms or conditions 
of change associated with artistic practice. 

4. Student Work: examples of past student work that addresses the concepts being studied at this time 
in the course. 

5. Links: Internet sites that address the ideas, issues, or related course information. 

6. Virtual Objects and Environments (QTVR): QuickTime virtual reality files are available for some 
assignments. The QuickTime plug-in must be installed on the student's Internet browser to access 
these resources. 

Critical Inquiry 

Students in the Ethnic Arts Online course are required to critically examine the interconnectedness of 
everyday representations, aesthetics, and culture. Another pedagogical aspect of the course was to 
facilitate student research of complex sets of lived events, contextual relationships, and how social issues 
contribute to the production, divergence, and shared meanings of visual forms of symbolic culture. In the 
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course, inquiry is simply defined as to direct ones immediate thoughts and actions about an idea, issue, or 
problem so as to increase one’s understanding of a particular solution or body of knowledge. However, 
students are asked to do more than just inquire about the “things” that people make special everyday. 
“Critical” inquiry is a continual process of identifying and posing problems, asking questions and 
questioning the validity of the questions asked. It entails the selection of inquiry processes to solve the 
problem or that are relevant to studying the ideas and issues. Students also learn that there can be multiple 
processes of educational and ethnographic inquiry. 

Inquiry is used to conduct ethnographic research about ethnicity, identity, representation, and visual 
culture. Course assignments provide ways for students to: share common interests and beliefs; discuss 
differences of interpretations; work alone and collaboratively on assignments; reflect on and critically 
think about the ideas and issues in relationship to themselves and within society; and make a long-term 
commitments to their own, one another's, and/or the group’s understanding of the online learning 
community. Students represent their research through formal and informal dialogue, structured and semi- 
structured questions generated by the instructor and student group leaders, and though written 
assignments. They are expected to conduct ethnographic research in their own communities, interviewing 
people about their “sense of peoplehood” or ethnicity and social alliances. This information is then posted 
on the course Web site and critically examined through various forms of communication spaces. 

Intercultural Communication Spaces 

Another goal of the course is to facilitate intercultural communication by building a supportive virtual 
community of online learning. This is achieved by clearly defining how technology can be used 
appropriately to complete assignments successfully. Our online course policy indicates a clear code of 
conduct for student participation in this virtual educational learning community. Technology is also used 
to create distinctive gathering places for group conversations and to encourage positive student leadership 
from within. Therefore to support student involvement and interaction, the Ethnic Arts Online course 
attempts to create a safe and dynamic virtual learning community. These kinds of intercultural 
communication spaces are becoming much easier with the aid of emerging technologies that allow for the 
design, production, administration, and maintenance of online messaging systems. 

Perhaps an example would be helpful to illustrate some of these points. In the first assignment students 
introduce themselves to a partner in order to facilitate community building among class members. They 
share these interviews online (interviews are an important aspect of ethnographic inquiry. Additionally, 
students are required to conduct an interview with a local artist and use the data in writing their major 
papers for the course). These introductions serve several purposes for creating an online class community 
including: building positive social relationships among class members, identifying common interests, and 
limiting the online group size to only a few people at the beginning of their online process of 
communication. Group collaboration is used throughout the course. As the quarter progresses, students 
participate in several varying group sizes of 2-6 members. Each member is scheduled to take an active 
role as a group leader for several activities, such as posing questions about the course readings, 
facilitating group schedules and activities, and helping fellow students probe life-centered issues of 
diversity, difference, ethnicity, representation, and aesthetics, as related to course content. Student 
involvement and interaction are mediated through the following synchronous and asynchronous computer 
messaging systems, or as I have referred to them in this paper, intercultural communication spaces (e.g., 
email, listserv, threaded discussion forum, chat room, and an electronic student portfolio). 

1. Chat Rooms: Chat room conversations happen in real time. Therefore, course members are expected to 
only use the chat rooms in small group assignments. In order to log-on at the same time, group members 
make prior arrangements for a common time to have their chats. 
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2. E-mail: There are two different forms of e-mail that we use in this course, individual email and a 
listserv. Students are enrolled in the listserv and any message they send to the “LIST” automatically goes 
to all of the class members. They are instructed to use the listserv for general correspondence between 
themselves, classmates, and the instructor. If they have personal business or are asked to send an 
individual email, this is achieved by sending the e-mail to the individual’s email address. Some 
assignments are sent directly to the instructor. 

3. Discussion Forum: The Discussion Forum is like a listserv with the exception that it is posted and 
maintained on the World Wide Web. Students use a browser (Navigator or Internet Explorer, not their 
email application) to access the Discussion Forums. One forum is created for each week of class during 
the quarter. The discussion forums allow course members to communicate with each other and to 
organize their conversations in one location on the Internet. 

4. ePortfolios: The electronic portfolio is a set of Internet web pages that consist of student papers with 
supportive images. The administration and forms are generated using Practical Extraction and Report 
Language (PERL). Students use this communication space to create their own online web pages during 
week 6, 7, and 8. In addition to text, these pages also contain digital images taken by students using a 
camera and transferred to a CDROM by a photo-processing store, digital camera, or captured from a 
scanner. Images are saved in the JPG format, siz^ no larger than 400 x 400 pixels @ 72 dpi. The images 
are then sent using the portfolio common gateway interface (CGI) form. The eportfolio Web pages are 
also used during the final group assignments during week 10 and 1 1 . After capturing information 
digitally, students post their ethnographic research in this eportfolio area, which is archived to preserve 
their inquiry for future classes. 

The creation of computer-mediated messaging systems is crucial to developing dynamic virtual 
intercultural communication spaces in distance. The integration of emerging technologies within online 
virtual education can offer students a less threatening way to become active participants in creating and 
making sense of visual culture. Virtual education allows students to use their time to explore, examine, 
reflect, and then write their interpretations through computer-mediated messaging systems. For many 
students, these virtual communication spaces help them feel more at easy. It provides them greater control 
over expressing their ideas with peers. Students in face-to-face classroom situations do not always 
experience this “comfort zone.” For example, language differences can create miscommunications and 
interpretation difficulties for some students. Both domestic and international students expressed that 
virtual intercultural communication spaces offered them a greater degree of security and confidence. Here 
I would like to note that the development of online learning communities should be continually examined 
for issues of conflict and the negotiation of divergent and shared values. The Ethnic Arts Online course is 
organized to facilitate interactions and collaboration among learners and the instructor. An adage we 
sometimes adopt in the course is that we should “agree to disagree” about some issues. 

Self-Directed Learning 

The course is also structured to create a self-directed learning environment for inquiry, communication, 
experimentation, and problem solving of ideas and issues. Self-directed learning is based on research that 
learners can actively create knowledge and meaning through direct experiences, reflection, manipulation, 
and critical inquiry of ideas in everyday life. Students are encouraged and required to participate in course 
activities and assist others with reflecting on and assessing the accuracy and application of their ideas. 
Collaboration, shared goals, and group work are powerful forces in this learning process as are group 
activities, simulations, and the use of open-ended questions. 

Criteria for all assignments are clearly represented and available online for students. Criteria are listed in 
different ways to address the diversity of different learning styles. The instructor reviews and grades all 
assignments, assesses the student's achievement and performance, as well as, helps students clarify 
assignment criteria and interacts with them as they pursue their research. The instructor serves as a 
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facilitator to the learning process by setting up: the use of collaborative assignments, forums for active 
discussions, and multimedia to aid in the development of critical inquiry and research skills. Students are 
directly responsible for exploring ways to successfully deliberate and engage in this form of educational 
practice. 

After students describe, reflect on, and assess their own learning performances, they are required to 
analyze the information they produce in relationship to other perspectives associated with the idea or 
topic. These culminating experiences are represented in their eportfolio presentations. Because students 
know that they are allowed to rewrite and resubmit assignments, they typically feel comfortable taking 
risks and their assumptions sometimes lead to incorrect choices. However, these risks more often lead to 
the generation of greater in-depth information. Student eportfolios and assessment often reflect adjusts to 
their interpretations learned through self-directed, peer reviewed, and critical inquiry methods. 

Summary 

Ethnic Arts Online requires students to effectively demonstrate achievement and performance with 
acquired knowledge in the course content areas. Student are not asked to recall or “plug in” what they 
learned. Instead, they must apply their knowledge with consideration of contemporary contextual societal 
conditions acquired through their own inquiry. Students are expected to achieve greater than the required 
minimal levels of participation. They must conduct research, which includes: collaborating with others; 
writing, revising and discussing papers; engaging in oral analysis of course ideas within the contexts of 
contemporary society; and discussing intercultural similarities and differences of opinions and 
interpretations. In conclusion, I believe that critical inquiry and integrated multimedia are some of the 
ways to strengthen the pedagogy of virtual education. Ultimately, the over-aching premise of the Ethnic 
Arts Online course has been to develop an online community that can support the needs of many different 
learning styles, improve student performances and instructional practices. This is a very large task and 
one that will need to be continually addressed in order to achieve success. Nevertheless, the purpose of 
this social constructivist approach is to support students not just being respondent to standardized 
curriculum knowledge or reflective of prior experiences. Virtual education and pedagogy should help 
students to seek out new knowledge, to empower themselves and others, and to be forward-looking in 
their life-long learning. 
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Third Generation Online Courseware: 

Don’t Replicate — ^Innovate 

Susan M. Kryczka 

Director, Office of Distance Education 
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Christopher B. Hughes 
Manager, Online Programs 
Boston University 

A majority of colleges and universities in the United States now offer educational programs online. The 
need to develop these courses quickly but at the same time satisfy academic considerations regarding 
their quality has produced a simple model for developing and delivering college-level online courseware 
— replicate the traditional on-campus course as closely as possible. Typically, this is done in one of two 
ways. The easiest is to use a basic learning management system available at most colleges, and have a 
faculty member produce an electronic text of the syllabus. Course notes, lectures, assignments, and the 
textbook of the comparable on-campus section are added, with a “chat” section for discussion or posting 
assignments. This “first” generation model for online course development is used for the majority of 
online courses now available. A “second” generation model has been developed for those schools 
equipped with broadcast or videotaping capability. Taped sessions of live class lectures are streamed on 
the web, with course notes, on-line discussion groups, and email, creating a video for online delivery. 

Both models offer the advantage of using existing course content thereby reducing faculty development 
time and preserving familiar course structure. Under these standards, distance education departments 
have been able to quickly and easily increase offerings with a common design approach at relatively low 
cost to the institution. 

But online instruction is no longer new. Universities can no longer be satisfied with what is primarily a 
low-tech, stagnant course product, housed in a high tech delivery system. As educators, we should 
provide courseware that improves on the traditional face-to-face learning experience. A “third 
generation” model is not only necessary but required at this stage in online development. 

New Assumptions 

Improvement means expanding our assumptions about online education: 

Online courseware does not have to imitate a traditional course to be an effective learning 
experience. The web offers the ability to create interactive modules and multimedia demonstrations that 
can be compelling and innovative without the physical presence of a teacher. The discussions which 
take place online offer a true “record of debate,” not just what might be recalled from a class discussion. 
We need our online courses to be valuable educational experience, not just the second choice to being in 
a traditional classroom. 

Developing online courseware requires a new model. The web is not the computer equivalent of a 
television, a delivery system for static programs. We should use the tools that the Internet provides to 
offer self-directed learning, interaction, and exploration. 

We must recognize different learning styles of adult learners and teach to that audience. We can 

incorporate adult learning into online courses. Adults learn by audio, video, reading, interaction, and 
testing — all active learning tasks. 
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Doing is what online courseware can be — instant feedback and problem solving. Approaching course 
development from the learner’s prospective helps to create interaction and a better learning experience. 

With these assumptions, Boston University set out to construct a “third” generation course development 
model. Our goal was to develop fresh courses for the web. While there may be a course delivered on 
campus with the same title, course number, and description and fulfill the same requirements, the 
similarity would end there. Faculty would contract with us in a different way and be a member of a team 
to develop the courseware. As subject matter experts only, they would not be required to worry about 
loading courseware, creating interactive elements, getting copyright clearances, be experts in 
instructional design, or anything else. Their most important function would be to think about delivery in 
a nontraditional manner, to provide content, review for accuracy, and be able to suggest multimedia 
strategies. The implementation of design and function would be the responsibility of others. 

New Goals/Standards 

The right development partners were needed to share this vision. Coming to the online world late gave 
us the advantage of learning from the successes and failures of others. From those lessons, we shaped 
our goals: 



❖ Develop new course content beyond that which can be delivered via text or straight video 
lectures. 

❖ Identify the unique qualities of the web and use them to support a new course development 
strategy through a team approach. 

❖ Build content made up of graphics, animation, print, audio, video and a wide range of 
published materials in a highly interactive environment. 

❖ Create high quality, scalable, easily revisable, courseware. 

♦> Incorporate student assessment and evaluation instruments and adult learning theory in the 
development process. 

❖ Measure student performance. 

❖ Established and meet objectives and goals suitable for the delivery medium. 

❖ Develop modules within courses that may have multiple uses as supplements in other courses 
or stand-alone noncredit programs. 

❖ Build courseware that results in a level of sophistication and high degree of student and 
faculty satisfaction based on performance outcomes, evaluations, and student interviews. 

❖ Further test the model by measuring completion and drop-out rates and revising courses based 
on the feedback from beta testing early courses. 

With these goals we knew that rapid production of courseware was not going to be possible. We focused 
on a team development approach and a few graduate certificates to begin with which met our design 
standards. We also prepared to commit resources, contract for outside assistance where needed and 
allow sufficient time for development. The University provided us with seed money to acquire the 
development time of faculty, as well as partner with outside specialists for instructional design, hosting, 
and course development. This was not an unlimited amount of money, but enough, if we budgeted right, 
to produce 6-10 courses the first year which would meet our standards. 

We felt that we’d need at least six months if not more to build the first courses. Our development team 
consisted of a subject matter expert (faculty), a hosting and design company, a cross platform learning 
management system, and a project manager/director within the university whose sole job it was to 
oversee the team and development of the course. The hosting and design company played a critical role 
in our success as providing not only the technical support but much of the design and vision for a 
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course. The University solicited proposals from two separate publishing/instructional design groups 
with specific experience in on-line course production. The company who won our contract was 
responsible for getting copyright clearances, Uye telephone 24x7 technical support (not just email), 
providing instructional design support and training our university instructional designers, and 
successfully translating a faculty member’s vision into a coherent effective, eleaming experience. 

We concentrated on graduate certificate courses in order to achieve a higher level of sophistication. The 
University administration wanted also to limit the early online experience to graduate or noncredit 
programming. We beta-tested courseware before we released our product for general enrollment. 

As with all plans, there were adjustments. We started to develop courses in late spring, with the idea to 
start to beta-test in the fall. Of the four courses identified, only one was ready for testing with a small 
number of students in the fall. The remaining three were still in development and postponed for offering 
until the following spring. We learned that though we were creating distance learning courseware, much 
of the development needed to be face-to-face with faculty and instructional designers, as well as training 
of the faculty. We also learned that a good agreement gets partners on board, but constant, consistent 
communication and project management is what keeps the schedule on track. And of course, the things 
that you never expect to go wrong, do, while the obstacles you anticipate don’t always materialize. 

As of spring, 2002 we had seven courses in a beta test phase which included graduate courses in clinical 
research from our medical school and a courses in instructional technology in education. In the Summer 
2002, we began a master’s degree program in criminal justice. In fall 2002, we will introduce the first 
course in a transitional doctorate degree in physical therapy. Our programs are interactive, vigorous, 
and, we believe, achieve our standards for using the technology as well as possible. They also require 
constant improvement as we receive feedback from student and faculty. These high-end programs have 
forced us to broadened our thinking, technical capabilities, our support structure and our vision. 

We have not yet achieved what we believe is the best “third generation” course with all the elements 
we want (and within our budget!) But we are close. We also have laid the groundwork for faculty 
to think in a different way in preparing to develop and teach on line. 

Of course, we’ve learned important lessons in the process. 

General Lessons 

Build a team that holds the same priorities and understands the goals of the ‘*third generation” 
course development model and communicate it often. We were careful in explaining our goals to the 
teams of faculty members (subject matter expert), instructional designers, a manager from the distance 
education office, and a project manager from the outside company hosting and assisting the 
development of the course. After careful review, we found a company who would provide us with the 
support necessary to create courses that went beyond the two models outlined and had a track record and 
understanding of where we wanted to go. We “sold” our ideas to colleges and departments within the 
University. We were honest about the of work required from faculty and that they would be part of a 
team, not sole contributor/owner of what would eventually end up on line. We also made clear what the 
University’s expectations were for the final product. 

Having established all of this, there was slippage in the schedule and members of the team who will 
misunderstand the commitment and goals. Reiteration of the goals as well as constant communication is 
required for success of the project. None of this was easy but necessary. 
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Provide a feculty agreement that is fair and generous. Boston University owns the finished online 
product in a typical work-for-hire arrangement. The University however has a royalty agreement in 
place to share revenue once the course makes a certain gross amount as well as money that flows to the 
. faculty member who developed the course if there is a derivative use, for example, sale of a few 
modules of a course as a stand alone to an outside company, etc. Payment for development of a course is 
generous. Faculty who feel they are treated fairly respond more positively to a process that requires 
hard work. 

Create certificate programs or common bodies of knowledge. It’s easier to sell a whole car, than 
parts of one, to both the person who will manufacture it or the person buying it. Get the buy-in of a 
department for an entire certificate or degree before starting the process. A commitment must also be at 
the dean’s level or the equivalent. Without a complete program, marketing individual courses becomes 
very problematic. 

Have one project manager who knows what all the members of the team are doing at any given 
moment in time. Developing third generation courses means managing lots of people on one product 
with a clear vision. Everyone thinks th^ know what a project manager is, but few people know how 
time consuming and detail driven it is. Having to go to three different people to find out where a course 
stands in terms of scheduled development, is not project management. Weekly conference calls with the 
instructional designer, project manager and design company and a weekly updated spread sheet of 
deliverables is necessary for all team members. 

Be prepared for more work and more frustration. Because we require a different type of course, 
“good enough,” isn’t good enough. An instructional designer may go back a number of times to a 
faculty member asking if a new simulation, diagram, or video is appropriate to add in a certain place. 
This is where creativity and appropriateness comes in. This means more time and effort on everyone’s 
part. 

Don’t assume money buys commitment or timeliness. Despite fair compensation and agreements all 
around, schedules still don’t get met for good and not so good reasons. Constant vigilance is necessary. 
Delays beyond a few days require communication and adjustments. 

We’ve learned from the recent demise of some University for-profit online entities, that even breaking 
even on these programs is a long-term goal. Boston University has accepted a slow growth model with 
the promise that it will be an excellent learning experience. 

We now have programs representing four separate colleges/schools and anticipate expanding to others. 
We also expect that other universities will continue to develop and improve their online courseware to 
reflect new technical and pedagogical goals. As we become more comfortable with the model 
developed and reach the standard we’ve set, we hope to decrease the development process to 4-5 
months. 

By building a “third generation” online model, Boston University hopes to contribute to the further 
improvement of online delivery. 
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CyberTower: 

A Model for Scalable, Sustainable Distributed Learning 



Diane Kubarek 

Project Leader and Instructional Designer 
Academic Technology Center, Cornell University 



Introduction: What Is CyberTower? 

What do gardens, engines, mastodons, and wine have in common? They’re all study rooms in 
CyberTower, a Web-based distance learning opportunity offered through Cornell's Adult University 
(CAU) and implemented by Cornell Information Technologies (CIT). To allow CyberTower to be 
accessible to the widest possibly audience, it is non-credit, very low cost, and requires a relatively low- 
end computer with network connection for participation. CyberTower's primary target audience is 
Cornell alumni. CAU offers both on-campus and travel study programs, which at full capacity can only 
reach 2,000-3,000 alumni per year. CyberTower is designed to reach out to the remaining 1 70,000-1- 
alumni who cannot participate in CALTs synchronous program offerings. In addition, CyberTower is 
being used as a recruiting tool for prospective students to introduce them to the diversity of subjects and 
faculty at Cornell, and as a distributed learning resource for incoming freshmen, who are required to 
begin their academic work during the summer before they arrive on campus. 

The Instructional Design Model: Making It Scalable 

Defining an instructional design model for a collection of course materials can provide flexibility, 
structure, and consistency, while streamlining the production process and promoting scalability. The 
instructional design model used for CyberTower provides structure and consistency but is adaptable for 
virtually any topic faculty may wish to present. In addition, existing rooms give faculty concrete 
examples of the end product, so they can be more focused during the process. 

Subscribers to CyberTower explore topics with Cornell faculty experts through multimedia lectures, 
presented in a format called a “study room.” Each study room is a self-contained mini-course. Faculty 
present their content via a streaming video lecture that is divided into sections, each a maximum of 12-14 
minutes in length. The lecture format is one with which faculty are familiar and comfortable, making 
entry into this new method of teaching easier. Each video is accompanied by synchronized visuals to the 
right of the video (see figure). These visuals can be still images, graphics, bullet points, text, or 
animations that illustrate the content. 

Each room has several additional supplementary sections. The Images section allows viewers to explore 
in more detail the images they saw in the lecture. The References and Links sections provide both print 
and Web-based resources that have been reviewed and recommended by the faculty, which viewers can 
use on their own to continue their study of the topic. The Discussion section of the room allows viewers 
to ask questions of the faculty, make comments, or talk about the topic of the room with other 
CyberTower participants. The discussion can be a vehicle for building a community of learners who 
share an interest and have direct access to the faculty. A final open-ended section can be adapted to the 
needs of the room; this section has been used most commonly as a timeline, a map, or a glossary of terms 

We have also enhanced the original instructional design model with a second lower-investment 
alternative called “forums.” A forum is a videotaped talk-show-style interview, lasting approximately 
one hour, on the faculty member's area of expertise. The forum is recorded live in a studio, eliminating 
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the need for video post-production. A single Web page presents the content of the forum, including the 
streaming video interview, a description of the topic, a brief biography of the faculty member, several 
recommended Web or print references, and a link to a discussion board for questions or comments for the 
faculty. 
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The Content Production Process: Making It Scalable 

CyberTower content is produced at a very rapid pace. Ten new study rooms and eight new forums are 
scheduled to launch each year. How do you go about producing new content at a rapid pace while at the 
same time maintaining an ever-growing body of existing content? For us, the keys to scalability have 
been the instructional design model that is well defined, a development team that is very small, and a 
production process that has well-defined stages. 

Each CyberTower room is assigned a room producer. All of our room producers are instructional 
designers and also have extensive experience with project management. The room producer partners with 
the faculty member from beginning to end, filling the roles of project manager, instructional designer, and 
video producer. Having one person fill multiple roles has many advantages; room production time is 
reduced because the person is very familiar with the content, and it also increases the faculty’s sense of 
having a “partner” in the process, someone who is equally invested in the work as they are. Our 
infrastructure also allows the room producers to build the room using relatively simple Web and 
QuickTime authoring tools. 

One audio/video professional on staff works on all CyberTower study rooms. The video professional 
must be very flexible in his or her approach to producing video; for example, some faculty will wish to 
script the lecture word for word, while others will speak comfortably from bullet points. . At Cornell, 
most faculty have never taught on video and have no experience in front of a camera. We shoot on 
location and only do productions that can be accomplished with a single camera to make the process less 
intimidating. We schedule a brief rehearsal with the camera several days before the shoot, and review the 
resulting footage with the faculty so they can be more comfortable with their on-camera presence. 
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Other contributors include a graphic designer to develop the room banners and navigation graphics, and 
student employees who assist with image scanning and preparation, creation of animations and 
illustrations, and data entry. If the faculty member wishes, they can hire a student who will be paid by the 
project to help locate supplementary Web and print materials. Librarians are also a valuable resource in 
this stage. Throughout, it is important to keep the faculty in the content expert role during the 
development process. The content is their specialty; everything else is likely to be new to them. 

Once the faculty member has finished writing the lecture, production of a room takes approximately three 
months. The first month is focused on recording the video, the second month on editing the video and 
preparing related materials, and the third month on synchronizing the visuals with the lecture and building 
the room itself 

It is also important to respect and acknowledge the faculty's other time commitments. We take a realistic 
look at the faculty’s calendar before room production starts and stress the impact of missed deadlines. 

The room producer needs to be an aggressive project coordinator, putting meetings on the faculty’s 
calendar well in advance, keeping the faculty moving forward, recognizing the faculty's working style 
early on and taking an active role in shaping the content of the room. 

CyberTower is not the only development project our ID&P group must manage, and we had to figure out 
how to juggle CyberTower with existing and new faculty development projects. .We have made it a goal 
to maintain a rigorous and regular production schedule so that we can plan this production work in with 
other projects. This is a goal toward which we are still working. We have learned that we need to recruit 
faculty far in advance - even years before work will begin. We need to recruit faculty in larger batches, 
and wherever possible, we need to have more content "in the works" than is on the production schedule, 
because faculty will delay and gaps in production will result. Toward this end, the CyberTower project 
leader oversees all room production, assists CAU in recruiting new faculty, develops and monitors the 
room production schedule, and coordinates room production with the overall ID&P production manager. 

Copyright remains a significant challenge to the project. We encourage the faculty to use copyright-free 
materials or materials they own in their room. Room producers collect source information on all 
materials from the faculty, and enter the information into a shared database. Staff at CAU work in 
collaboration with librarians to identify copyright holders and request permission. A newly formed 
Copyright Advisory Committee comprised of librarians and lawyers from University Counsel’s Office 
make legal recommendations when needed. However, the number and complexity of the copyright issues 
arising from CyberTower continue to grow, and clearances often take long periods of time to obtain. 

The Infrastructure: Making It Sustainable 

Long-term sustainability of any distributed learning application depends on having a scalable 
infrastructure to store and deliver content. CyberTower was originally developed using a combination of 
static Web pages, Microsoft Access, and Active Server Pages (ASP). However, we knew this 
infi-astructure would not scale to large numbers of users or large numbers of study rooms; maintenance of 
the content would become increasingly time-intensive, and performance of the application would degrade 
over time. We therefore redesigned and redeveloped CyberTowefs infrastructure from the ground up 
using Java and Oracle. Again, the use of a consistent instructional design model enabled us to make the 
site dynamic and almost completely database-driven. Over a period of about nine months, one lead Java 
developer worked closely with a code review team to design and build the database and code structures. 
This new cross-organizational collaboration was a valuable professional development opportunity for all 
involved, from the lead developer still early in his career to the most seasoned Java programmer on staff. 
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Though the up-front investment was significant, the benefits to building this infrastructure are numerous. 
The process of building rooms has been greatly streamlined, cutting down greatly on room production 
time. We knew that in the old infrastructure, maintaining 30, 40, or 50 study rooms would have become 
very time-consuming. Now, maintenance of the growing body of content is practical since there is no 
longer duplication of content in static Web pages. The new infrastructure is robust and scalable; it will 
support far more simultaneous users than we ever expect to have. The application was designed to 
maximize performance, so the user's Web browsing experience is much faster and much improved. The 
old infrastructure also represented aging technologies in which our organization had no existing expertise; 
the cost of developing this expertise for a single application would have been significant. 

Content storage is not the only infrastructure required to offer an online distance learning product. Just as 
critical is providing online registration and account management. Today's distance learning audience 
expects instant e-commerce access to offerings, and the system will not be scalable if staff have to be 
involved in this never-ending administrative function. In our case, we were able to tie into CAlTs 
existing online registration system, making CyberTower one "course" in our School of Continuing 
Educations' offerings. The School's technical staff collaborate with CIT to enhance the registration 
system as needed. 

One notable lesson learned was that there are often multiple acceptable solutions to technical problems. 

A critical juncture in the infrastructure redesign was choosing between MySQL and Oracle for a database. 
For us, the choice was more of a resource one than a technical one, since by using Oracle we can purchase 
database administration services from the group that specializes in this work. Another example of such a 
choice is the streaming video technology being used. Though CyberTower is currently committed to 
QuickTime streaming, we must keep abreast of technology developments and periodically re-evaluate 
that choice. Junctures such as this illustrate the need for an overall project leader who can oversee the 
process, facilitate collaborations among different groups, develop the infrastructure production schedule, 
and set criteria by which technical solutions will be chosen. 

User Support: Finding an Appropriate Model 

CyberTower has a potentially huge audience of users with a very wide range of computer hardware and 
level of comfort with the technology. Before launching to the general public, we opened the product to 
about 1 50 alumni who had volunteered to be beta testers. All participants received a Getting Started 
packet in the mail with instructions for setting up their computers. We established a dedicated phone line 
and a special email address, both monitored by talented student employees, similar to our Helpdesk for 
Cornell's residential students. It quickly became apparent that such dedicated support was not necessary; 
the bulk of the questions came in the first couple of weeks and reduced to almost none thereafter. The 
nature of the application was that users needed help when they first subscribed, but not afterwards. 

When we launched to the general public in the summer of 2001, we modified our user support model 
based on our pilot experience. We eliminated phone support but kept the email address, and folded the 
monitoring of that address into the standard procedures of our residential Helpdesk. From that 
experience, we learned that case resolution time became extremely lengthy; users would often take many 
days or weeks to reply that a proposed solution had or had not worked for them. We were unable to help 
users "in the moment" that they were having the problem, and the product is not of such a pressing nature 
(i.e. not for credit) that the client is likely to follow through to a solution. These two factors combined to 
create a client perception that the product does not work. 

In response to this experience, we reinstated phone contact, but rather than set up a dedicated line, 
CyberTower became one of the many products supported through our residential Helpdesk's main phone 
line. The immediacy of phone support provides better service to those who need it, and perhaps more 
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importantly, client perception of better service. We continue to monitor case resolution times and demand 
for user support as the number of subscriptions rises, and expect to modify the model fiirthei’ to better 
accommodate user expectations and needs. 

After the general launch to the public, we received feedback from potential subscribers that they did not 
want to pay for a subscription until they knew whether their computer hardware and network connection 
would deliver the application satisfactorily. In response to this feedback, we implemented an automated 
sign-up for a free three-day trial subscription. The trial allows users complete access to the application 
along with access to our user support helpdesk, so they can know their computer will deliver CyberTower 
before paying the yearly fee. 

University-Wide Collaborations: Don't Go It Alone 

Very early in the development of CyberTower, we realized that the project's stakeholders represented 
many organizations across the University. Looking beyond our central IT organization for partners and 
forging new relationships has brought wide acceptance of CyberTower and a sense of mutual investment 
in the project. CAU owns and promotes the product, maintains the registration system, and recruits 
faculty. CIT supports the infrastructure and works directly with faculty to produce the content. Cornell 
University Libraries assist with content research and copyright concerns. Media Services, a video 
production service on campus, produces the multi-camera live-switched video for forums. University 
Counsel provides legal guidance for copyright concerns. And of course, the faculty themselves provide 
the content for the site. Forging new relationships among so many stakeholders has offered long-term 
benefits which extend far beyond the CyberTower project. For example, a new committee on copyright 
concerns at Cornell was formed partly due to the new demand for clearances generated by CyberTower. 

Just as challenging was establishing brand new working and financial relationships between different 
areas within CIT. In order to obtain all the necessary technical expertise, staff were drawn from four 
separate divisions: 

1. Distributed Learning Services (Academic Technology Center): Project management, instructional 
design, single-camera video production, lead Java developer, content production 

2. Integration and Delivery (Administrative Systems): Java code design and review team, Oracle 
database development and administration, QuickTime streaming server administration 

3. Customer Services and Marketing: online and paper documentation, user support 

4. Systems and Operations: NT and UNIX systems configuration, hardware specifications and 
standards 

These working relationships continue to evolve and reappear in other instructional technology projects 
managed by the Academic Technology Center. 

Biographical Sketch 

Diane Kubarek is a project leader and instructional designer with Cornell Information Technology's 
Academic Technology Center, the central support organization for faculty using technology in teaching. 
In addition to leading CyberTower, she consults with faculty on various instructional technology projects. 
She assists in designing curriculum and teaching hands-on workshops for faculty. Recently she led an 
evaluation of online testing software products for possible implementation campus-wide. 
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Email: 

URL: 

Phone: 

Fax: 



126 CCC Garden Avenue 
Cornell University 
Ithaca, NY 14853 
dmk2@comell.edu 

CyberT ower: http://cybertower.comell.edu/ 

Academic Technology Center: http://www.cit.comell.edu/atc 

607.255.1290 

607.255.4479 
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Retail Data Mart Online Course: 

“The First-Best Online Training Experience” 

Nancy Laich, USPS 
Executive Program Director, 

HR/Payroll Enterprise 

Gail Breed 
Operations Specialist 

USPS Field Operations Standardization Implementation 
Background 

With the introduction of the Retail Data Mart (RDM), the Postal Service can use Point-of-Service One 
data to track and analyze customer preferences and purchasing trends. Data transfers on a daily basis 
from the terminals at the post offices to the Retail Data Mart where reports are available for access by 
managers in these offices. 

Prior to November 21, 2001 training on the Retail Data Mart (RDM) was conducted in a classroom 
setting using contracted instructors. About 30% of those trained in this traditional method used the 
available reports. Retail Workforce Strategies (RWS) led by Nancy Laich, was tasked to produce an 
online self-study course so that the costs associated with travel and replacement hours could be saved. 



The benefits of online training include: 

❖ Elimination of travel and employee replacement costs 

❖ Immediate testing of knowledge 

❖ Working at one’s own pace within a reasonable timeframe 

❖ Voice Of the Employee (VOE) training hours credit 

Pilot Results 



The Pilot and Goal 

The RDM Online Course was piloted begiiming November 21,2001 with 263 registrants from six 
different geographical areas. They were ready to test the effectiveness of the end-to-end development and 
implementation of online training for the Retail Data Mart (RDM). They were accessing a completely 
new system for automated registration with course access 24 hours a day-7 days per week that included 
an automated course scoring and certification. This is a first for USPS. Eight different groups of people 
coordinated in this work to go from concept to implementation. These included 4 postal groups (RDM 
Program Office, Retail Workforce Strategies, and Information Technology) and 4 contractor groups 
(NCR, MicroStrategy, Zerone and Northrop Grumman Information Technology). 

The goal of the training was to enable each participant to be ready to generate and use RDM reports in the 
workplace to address one or more of the following strategic retail goals: 

❖ Remove low-value transactions from the window (stamps only) 

❖ Identify revenue opportunities such as value-added sales the manager can influence and 

❖ Identify how the work budget can be more effectively manipulated against actual workload. 
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The Results 

Delivering this training via the web, as opposed to the classroom has resulted in cost savings estimated at 
$1.5 million. Reduction over classroom costs exceeded 85%. 

The online course templates are re-usable for other Postal courses at less cost than a completely new 
development 

❖ The current 103 graduates of the course have tested successful in the techniques and business- 
related problem solving with an average score of 94%. 

«I« 70% of the graduates are accessing the RDM. 

❖ This first best experience with online learning is an enterprise wide change management 
application for postal operations 

Who’s Who: The Population Demographics 

One hundred seventy people registered to take the course with 103 completing the course. Of those who 
completed the course and accessed the RDM, 72% accessed it on the same day they graduated from the 
course. 
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While the course was designed with business scenarios for Postmasters and Supervisors, a wide variety 
of USPS employees took the course to learn the techniques required to access reports from the Retail Data 
Mart. For example, employees from the Inspection Service, and various groups within Retail and 
Customer Services requested to participate. Several other groups had a single student, as in the case of an 
Information Specialist Technician, Manager of Operations Support, secretary and a letter carrier. 

Test Scoring 

In addition to the lessons, the activities were designed to include a business-related scenario using a 
particular report. For example, lesson three teaches to the Stamps Only Report and controlling inventory. 
The corresponding activity tests the user knowledge and skills in the techniques as well as the business 
application of the reports found in the Retail Data Mart. 




These successful scores by the graduates demonstrate a high level of understanding of the use of the 
reports available within the RDM. No longer is there training for its own sake, but measurable data 
related to the training can be tracked between on the job performance and the teachings from the course. 

You may now be asking if there was any real deviation, the average participants completed well above the 
passing rate of 75%, as a matter of fact the average score for all activities was 94%. The standard 
deviation is .035. 



Days to Completion 

Now that we know who’s who and how well they scored, the next question was how quickly they 
completed the course. The postal inspectors were treated as a group while everyone else was grouped by 
area location. This distinction was made because the postal inspectors are currently the single greatest 
users of the Retail Data Mart. 

On average, a participant registered and then completed the course over 1 8 days. However, 19% of the 
time, these graduates registered and completed the course within one day. 

When the data was charted, the trend lines are identical, indicating no difference in the number of days 
each group spent in the training. 

Co mmuni cations and Registration 

Interest in registration was driven by several variables: 
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1 . Timely and regular messaging to each area about status of enrollment 

2. Providing certificates of completion to graduates 

3. Establishing an end date to registration and completion 

Below you will see the big picture of total registration for the pilot. 
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Several key messages occurred over the course of the pilot. The first was the “Go Message” sent 
1 1/21/01 to the area coordinators notifying them about the process to register via the web site for el357. 

The next key message occurred on 12/10 announcing the first three graduates of the RDM Online Course. 

Biweekly status reports began in 1/14/02 and provided details on which people for a given area had 
registered and completed the course. 

Certificates of completion were also sent for the remainder of the pilot, until its conclusion. 

The final message on 3/19/02 included the announcement about ending the pilot on 3/22/02 along with 
the status update. 



Testimonials 

These are the testimonials from the graduates of the online learning regarding to how this online course 
positively affected job performance. 

❖ Like, Ed, I think the data is excellent to be used by each PM nearly weekly or at a minimum 
each AP. For us it is a mother lode vein for finding out days, times and volumes of products 
sold — such as Express Mail — if I am correct we could even find out the destinations of the 
pieces. 

❖ The training on-line is very effective — ^much more so than in a classroom without hands-on 
training. Because it is self-paced and can be done in your office you learn more. Also, the on- 
line training is available always — so if you need a refresher you can retake a short module to 
find the reports you need. 
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❖ Susan took the first training without the on-line course and said she learned more through the 
on-line course. 

❖ The course was created with Postmaster input throughout to make it easy to use, quick and 
continuously available. They even made it easy to figure out which reports would be of more 
value for Postmasters and be able to store their own canned reports so they can just pull them 
without setting it up again. 

♦> The course is very practical. It allows the trainee to learn at his/her pace. I appreciate the 
consideration built into the program that makes a reasonable allowance for those who have 
difficulty taking tests. 

❖ More training and certification should be offered in this web-based format. The program allows 
the user to open the course at any time that is most convenient. 

❖ The RDM format allows users to learn how to manage and set up strategies to get the best 
results out of existing data. No data outlet that I am aware of allows the user this benefit. 

❖ The RDM course format should be the standard by which all other data outlets are designed, 
and all should be accessible from one Web location. 

♦> I am a secretary just pulling reports for the Postmaster, so it hasn’t affected me, but I can 

certainly see how the reports could help managers & supervisors, particularly in defining which 
sales associates are selling which products better and in seeing how many transactions an SSA 
handles compared to their peers. 

❖ I really haven’t had a lot of time to search around on the RDM to see how it has helped. I think 
that it will be good for me to spend some time and play around with it a bit to see exactly what 
useful information there is. 

❖ The CBT RDM course was excellent. I really liked the on line cheat sheet. This simple 
handout gives a good definition of the reports and a drill path. The basic reports are an 
excellent starting point for field users. Thank you for letting me take this course. I would 
encourage others to do so. 

♦♦♦ Though I haven’t had a lot of time to take advantage of this new (to me anyway) source for 
useful information, I can see it being a helpful tool to research many different types of retail 
data. 

❖ The Online Course provided a refresher on the various reports available in RDM. I believe this 
is an excellent way to update the field as enhancements are made to RDM. Additionally, when 
HQs identifies certain reports everyone should be using, they could provide CBT explaining the 
report and how to access it, the path to use. This is a great tool to use in support of the RDM. 

❖ There is so much information to be obtained from the Retail Data Mart. I know that this will be 
an invaluable asset to future retail planning in our office. 

Biographical Sketches 

Nancy Laich is an executive program director with the United States Postal Service in Employee 
Resource Management. In her current position she is responsible for the implementation of process 
redesign and system configuration of and Enterprise Resource Planning system for all human resource 
functions. This effort will impact almost 1 million career and temporary postal employees. In her former 
position as Manager, Retail Workforce Strategies, Nancy worked with a team to design a process using 
distance learning that is covered in this workshop. 
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E-Mail: 
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Phone: 


202.268.2878 
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202.268.4321 
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Gail Breed is an operations specialist with the United States Postal Service in Field Operations 
Standardization Implementation. Ms. Breed is responsible for the design, development and 
implementation of an online course for processing operations. Ms. Breed has an undergraduate degree 
from James Madison University and began working in the USPS in 1989. She has demonstrated 
exceptional organizational and managerial skills which have resulted in increasing levels of responsibility 
with projects of nationwide scope. 
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Learning by “Doing” and “Experiencing”: 
A Success Story 

Dan Lim, Ph.D., Assistant Professor 
Director of Instructional Technology Center 
University of Minnesota, Crookston 



Learning by “doing” and “experiencing” is a lot more feasible and productive when highly interactive 
learning technologies are well integrated into the curriculum. This paper describes the background, 
design and delivery, and outcomes of the Summer Institute of Instructional Technology held last summer 
(2001) at the University of Minnesota, Crookston. The intent of this paper is to share the rewarding 
experience in using a constructivist learning approach in conjunction with technology modeling to train 
selected faculty from Singapore and Minnesota to be certified professionals in instructional technology. 

Background 



Origin 

After giving a keynote address at a national symposium on technology in Singapore in September, 2000, 
the author was asked to find some technology integration courses to help train some 60 Educational 
Technology Champions from among 1200 educators at the Institute of Technical Education (ITE) of 
Singapore. The Institute enrolled about 18,000 students each year. The administration of the Institute 
was embarking on a 5-year technology master plan to provide web-based learning for on-campus and off- 
campus students in Singapore. The proposed 60 trained Educational Technology Champions would be 
the critical core group helping to train the entire faculty to adopt and develop online courseware. Since 
the administration of the Institute wanted to emulate the philosophy, instructional design, and technology 
integration model at the University of Minnesota, Crookston (UMC), the author proposed the concept of 
the Summer Institute of Instructional Technology to be held at UMC. At the end of the training period, 
the participants who successfully completed the course requirements would receive the Instructional 
Technology Certification from UMC. 

Rationale and Concept 

The concept of a Summer Institute of Instructional Technology was conceived to meet the proposed needs 
and requirements for training selected faculty to become educational technology champions for both 
Singapore and Minnesota. The main rationale for conducting the summer institute was to train a core 
group of “trainer” educators who will in turn help train other educators to integrate technology into the 
teaching curriculum. An onsite hands-on training of 5 weeks during summer on instructional design, 
development, and technology integration would provide the participants a total “immersion” of industry 
standard and emerging instructional technologies. Completing all course assignments, projects, and other 
requirement online and at a distance would provide the participants the authentic experience of learning, 
managing, troubleshooting, and interacting in a “faceless” environment. 

Planning 

The process of organizing and preparing for the summer institute took about 6 months. Content analysis 
was conducted among Singapore and Minnesota participants. The selected course contents were 
integrated into 6 courses: Introduction to Instructional Technology, Instructional Design & Development, 
Graphics & Interface Design for Courseware, Courseware Development & Management, Computer-based 
Instructions & Training, and Curriculum Integration of Technology. A thorough learner analysis was 
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conducted among the participants and their background, experience, motivation, and reading level were 
taken into considerations in designing and developing the courses. The logistics of providing laptops, 
software, logins, and WebCT setup took about a week to complete for all participants. 

Philosophical Framework 

The underlying philosophical framework for the Summer Institute of Instructional Technology was using 
a constructivist learning environment to help participants apply technology integration and develop 
authentic and well integrated learning objects. A “guided” constructivist learning environment 
transformed “the role of the learner from receiver (classic, communications conception of learners) to 
producer, creator, and sender” (Jonassen, 2000, p. 24). The major learning outcomes of courses were to 
empower the participants to create technology-based learning products that were authentic and had strong 
contextual integration. 

Modeling of technology integration was a vital component of designing the summer institute courses. 

The intent of the author was that the participants would emulate the use of highly interactive technologies 
demonstrated and experienced throughout the duration of the courses. Jonassen (2000) pointed out that 
when technology is used as a constructivist learning tool, it “can help to transform learning and leamers- 
to help them to become independent, self-regulated, life-long seekers and constructors of knowledge” (p. 
25). Participants were expected to be responsible for constructing their own learning experiences and 
development using the learning opportunities, collaboration with peers, technology resources, facilitation, 
support provided by the instructor, de Caprariis (2000) noted that “Jonassen’ s point — ^that students who 
take charge of their own education, who are no longer assimilating ‘canned’ instruction, and are thus able 
to construct their own version of reality — s worth considering” (p 4 1 ). 



Design and Delivery 



Interactivity: “Doing” 

Although this Chinese saying “What I hear, I forget; what I see, I remember; what I do, I understand” has 
been widely quoted, the concept still challenges most educators in making learning interactive for 
students. What mesmerizes millions of young people around the world about video games is its high 
degree of interactivity. When some interactivity is introduced to an absolutely passive lecture, students 
become more interested in the content presented. The key of making education interactive is providing 
students with more learner control. 

The design of the Summer Institute of Instructional Technology courses followed a formula of 30% 
passive content and 70% interactive (application) content. The 70% interactive content consisted of 
interactive simulations, job aids, practice, reviews, and self-assessment. Providing creative feedback in 
interactive application content was vital in motivating students and reinforcing content retention. The 
participants found the interactive applications motivating, easy to follow, easy to comprehend, and most 
importantly, achieving better retention of what they were learning. 

Participants at the Summer Institute of Instructional Technology learned by doing interactive practice and 
reviews (to learn intellectual content), interactive simulations (to learn skills and techniques), handy job 
aids (to perform specific tasks), interactive learning games (to help retain difficult or complex content), 
and hands-on creation of learning applications. Presentation content and instructions were presented in 
small chunks (in HTML and PowerPoint) to guide participants in doing the interactive and hands-on 
activities. Figure 1 is an example of an interactive practice/review (drag and drop application) to help 
participants learn the criteria of determining whether to use CBI/’CBT or group instruction. Figure 2 is a 
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Flash learning game designed for the participants to learn and remember the terms and concepts used in 
the third course: Graphics and Interface Design for Courseware. Figure 3 is an interactive simulation for 
prioritizing learning tasks. 



When To Use CBI/CBT 
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Figure 3 



Modeling : “Catching” 

There are two important reasons why technology integration should be modeled in teacher technology 
training courses. Firstly, the nature of the program, which is instructional technology, requires a certain 
degree of technology integration. It is more effective to convince the participants to integrate technology 
into their teaching when they see and experience a high degree of technology integration themselves. 
When they are excited by the high degree of technology integration, they are more likely to use 
technology when they return to the classroom. Secondly, students tend to teach the way they are taught. 
Modeling technology in the courses will subtly help them carry it over to their teaching career. 
Compelling software, highly interactive web-based technologies, and collaborative learning systems were 
integrated into the courses. Technology modeling was implemented through the use of Toolbook 
Instructor, Macromedia Flash, Adobe PhotoShop, Adobe Illustrator, Webcrossing, WebCT, and other 
software. The author used these compelling software to provide participants with compelling visuals and 
interface, interactive learning objects, and a single convenient interface for all courses. 

Constructivism: “Experiencing” 

As most learners learn constructively with most hands-on tasks, it makes sense to create a constructivist 
learning environment for all courses in the Summer Institute of Instructional Technology. During the 
onsite hands-on classes, participants practiced using various software to create learning applications. 
Visual tutorials and virtual labs were provided to help participants to self-leam or review. When 
participants completed all projects and course requirements, they would have had “experienced” 
technology integration in a constructivist learning environment by being in the program, practice, 
creating, troubleshooting, and producing interactive learning products for their students. During the 6 
months of distance and online learning, participants actively discuss in WebCT bulletin boards answering 
questions posted by the instructor and exchange views with fellow participants. They posted daily (5 
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times per week) logs documenting their struggle, progress, troubleshooting, and reflections. It was very 
rewarding to see fellow participants responding to each other’s logs affirming ideas and helping each 
other out. 

Outcomes 



Observations 

The author observed a great change of attitude towards the constructivist learning approach, especially 
among the Singaporean participants who claimed that constructivism would never work in Singapore. 
Throughout the duration of the distance mode of learning, participants became more self-reliant in using 
and troubleshooting technology. As all course projects were derived from their workplace, all 
participants had used what they created in at least one class. The most exciting part was evaluating the 
reusable learning objects they created. Many went beyond the instructor’s expectations. Another 
exhilarating moment was seeing participants emulating the way technology was modeled in their course 
projects and teaching experience, especially online teaching. There were several downsides: technical 
support was not adequate for Singapore participants; a variety of examples (for various disciplines) was 
not available; six months could not do justice to the required course projects. Here are some excerpts 
from the daily logs of the participants: 

Sample Excerpts From Daily Logs 

“One of Dr. Lim’s assignments was to write a paper on what we had learned as constructivist online 
educators. My paper was accepted for a national conference.” 

“The good news is that I have made progress and my progress is documented by my class journaling (a 
very good thing to use in any class). I smile as I continue to search for that pot of gold at the end of the 
rainbow. I feel a little like Dorothy, from The Wizard of Oz, taking one step at a time on the yellow brick 
road journeying toward the wizardly wisdom of ITC.” 

“I did a kind of ‘soul searching’ after reading Dr. Dan’s reply. What struck me most was the phrase 
‘learning context’ mentioned. I had another look at my task analysis and task structure plan and finally 
managed to come up with not one but six learning frames for my first subtask of my first LO (learning 
object) ... As usual there was an initial struggle. I did not seem to know where I was heading until my 
wife pointed out that our son was not learning his spelling. It struck me that he was not enjoying the 
normal practice of rote learning. I devised a simple game of matching, words scrambling, word 
arrangement and fill-in the blanks of spelling words. They were the same words put in different 
‘situations.’ I began to appreciate the need to ‘overteach.’ In no time my son was able to learn his 
spelling well. When I started to work on learning frames for the first subtask, I realized the need for a 
‘learning context’ in my CBI development.” 

“Browsed the content put up by <deleted> in ITC540. Attempted some of his quizzes and tests. It was 
interesting and exciting going through the course. <Deleted> has done a good job in ensuring that the 
students stay on his course. Well done, <deleted>. Hopefully I can create a module as excellent as his.” 

“Prepared answers for the questions posted in the discussions. Questions in the discussion were very 
useful for me to recap and reflect on my personal experiences and thoughts while going through the 
module.” 
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Providing Direct Service Distance Education 
to Students With Disabilities. 

Lori Magnuson, M.A. 

Training Specialist 

Center For Continuing Education In Rehabilitation 
Western Washington University 

Conrad Kuehn, M.A. 

Technical Specialist 

Center For Continuing Education In Rehabilitation 
Western Washington University 

Introduction 

In October 2000, the Center for Continuing Education in Rehabilitation (CCER) at Western Washington 
University began a distance-education based program funded through the United States Department of 
Education Rehabilitation Services Administration to prepare people with disabilities for careers in the 
human services field as Direct Service Professionals. Over the five year period of the Direct Service 
Professional (DSP) Project, we anticipate an impact at three levels: 1) assisting 100 individuals with 
disabilities in becoming employed as Direct Service Professionals (DSPs), including self-employment if 
desired; 2) assisting 400 other individuals with disabilities and other needs to develop and implement 
employment and/or independent living goals through the efforts of the CCER-trained DSPs; and 3) 
incorporating the DSP service option into the activities of state and Native American VR agencies, one- 
stop career centers, and other disability service agencies. The training curriculum, student marketing 
strategies, and DSP employment approaches will be disseminated nationally. 

The disability community is a large part of the total population. According to 2000 data from the 
U.S. Census Bureau, there are 22.2 million people with disabilities of working age in the U.S. 

Despite efforts to bring people with disabilities into the mainstream of education with IDEA 
(Individuals with Disabilities Education Act) and the mainstream of the workforce with the ADA 
(Americans with Disabilities Act), more than half (51.3 percent) of these individuals are 
unemployed. 

People with disabilities are particularly underrepresented in professional occupations, including human 
services. This five-year demonstration grant was designed to explore distance education as a method of 
bringing applied, career-focused education to people with significant disabilities, particularly individuals 
from communities of color and from rural areas with limited access to training and educational 
opportunities. The lessons learned from the project are applicable to any distance education program 
concerned with providing access and support to students with disabilities, and other non-traditional 
populations. 

Several challenges were encountered in the educational design for this project. The content, materials, 
and approaches had to be designed for students with on-line experience that varied from no experience to 
expert; and academic experience ranging from high school to the graduate level. All the students have 
single or multiple disabilities such as quadriplegia, blindness, hearing impairment, mental illness, 
learning disabilities, and other barriers that challenge major life activities. Through the Internet-based 
course work, amplified with sbc days of on-site training; conference calls; and intensive, individualized 
technological and content support; students learn both the technical and soft skills needed to be 
successful in a human services career. 
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Our target population is modeled to be rural and serve communities of color and others without access to 
instructional facilities, or with other educational access problems. Included in the two cohorts of students 
to date are 19 females, 14 males, 13 from rural areas, 8 from small towns, 1 1 from large cities. Five 
students are Native American. Disabilities include; hypothyroidism. Bipolar disorder; orthopedic 
impairments; drug and alcohol addiction, degenerative joint disorder; vision impairment, schizophrenia; 
explosive disorder; generalized anxiety disorder; paranoia, depression. Attention deficit disorder, 
quadriplegia, brain injury, carpal tunnel, joint degeneration, hard of hearing. Fetal alcohol effect, schizo- 
affective disorder. Crone’s syndrome, fibromyalgia, specific learning disabilities. 

The Project 

The project crystallized into two main categories. The first was the design of the instructional product. 

The second is the technology needed to deliver the instructional product. The common denominator in the 
design of this program was to reach an audience that would run the gamut from skilled and technically 
experienced to unskilled and a non-technical penchant. Two cohorts have been completed and two more 
are in the recruiting stages with a total of sbi cohorts planned. 

Instructional Approach 

The instructional approach is to find a general methodology that allows the students to fins a way to learn, 
share their learning and to find a sense of community amongst their fellow students. Two aspects have 
become the focal points of the program. The first is the face-to-face meeting and the second is the 
extensive use of the asynchronous discussion group. 

Two face-to-face meetings are designed with the first to take place after the students receive their 
technology and before the first Internet-based courses lessons begin. In this three-day session held in a 
central location to the area covered by the students (First Oregon-Eugene, Second Alaska- Anchorage) the 
emphasis is on project overview, technology training, and community building. 

The second face-to-face takes place before the two final Internet-based courses. This is held in the same 
location as the first and the emphasis is on further community building, internship and employment 
search skills. Panels of employers and human service providers are available for information and 
discussion. Interview techniques and practice are also provided. 

Asynchronous discussion groups are formed and become the main platform for learning. Assignments are 
posted, and students are required to submit homework assignments as new threads in the discussion board 
and student comments and replies are required. This discussion process is the main exercise of learning in 
the online environment. 

The curriculum was developed by the project staff and is constantly being updated and modified. The 
third cohort will be a Native American group, and Native consultants were brought in to review and 
advise on curriculum modifications. Constant feedback is elicited from the participants to facilitate the 
curriculum modifications. 

In addition to the curriculum materials developed by project staff, we are using the text Human Services? 
That Must Be So Rewarding, by Julia Bernstein. We are also using Disability Handbook by the 
Department of Rehabilitation Education and Research at the University of Arkansas as a text for the 
course on Disability Information and Resources. A number of elective courses are offered in collaboration 
with the Continuing Education Center at the University of Wisconsin-Stout. 
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Original Courses and Order 


Current Courses and Order 


Month 1: 

1 . Principles and Practices of Effective 
Communication 


Month 1 (includes on-site days): 

1 . Developing Relationships (title change - 
content is the same) 


2. Principles and Values of Informed Choice 


2. On Being a Direct Service Professional 
(title change from Self and Client Advocacy- 
content is the same; order change from third to 
first month) 


Month 2: 

3. Understanding Disability Information and 
Resources 


Month 2: 

3. Understanding Disability Information and 
Resources 


4. Role of Family and Support Systems in the 
Rehabilitation Process 


4. Principles and Values of Informed Choice 


Month 3 

5. Self and Client Advocacy 


Month 3: 

5. The Role of Families and Support Systems in 
the Rehabilitation Process 


6. Cultural Diversity 


6. Cultural Diversity 


Month 4 

7. Negotiation and Conflict Resolution 


Month 4 (includes on-site days): 

7. Negotiation and Problem Solving (title change; 
content is the same) 


8. Group Facilitation 


8. Group Facilitation 


Month 5: 

9. Community Resources 


Month 5: 

9. Federal, State, and Community Resources 


1 0. The Role of the PRC: Working with 
Professionals and Other Support Personnel in 
Organizations 


10. Working with Professionals and Other 
Support Personnel in Organizations 



In addition to completing the ten core courses, students complete 120 hours of work experience as direct 
service professionals in their local communities. 

Technology 

This aspect of the project has presented many challenges. Our commitment was to bring Internet training 
to anyone in the targeted region or state who fit within the criteria of the grant. The only barrier directly 
related to technology would be the ability to have Internet access in the individual student’s home. The 
technology requirements are a 28k modem or better, a computer with Internet Explorer, and the ability to 
use the Blackboard course presentation courseware. This eliminated some online services such as AOL, 
though with version 5 of BlackBoard, compatibility has improved. The last requirement was some type of 
word processing program for ease of posting to the discussion group. If Microsoft Word was not 
available, a free text editor such as NoteTab was used. 
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The students are contacted by the technology specialist to determine their technology status and needs. 

The project technologist who trains and set up the workstations at the participant’s houses visits most of 
the students who do not have computers. This enables him to determine the skills and need levels of the 
students in their home contexts. The technologist also supplies ongoing help services throughout the 
duration of the Internet courses. 

The technology survey is given on-line or by phone or in person if necessary. For the initial cohort, nine 
computers were purchased and eight were delivered. The skill levels for the first cohort were: three 
students who had never used computers, four who had modest computer skills, and one student with good 
experience and skills. Some students needed Internet access, while two others needed either partial 
equipment or upgrades on some of their existing equipment. 

Students with no computers or computers with obsolescent technology were supplied with loaned 
computers and printers for the duration of the coursework and work experience (approximately 8 
months). The computers are 700MH Pentiums withlO gig hard drives and 56k modems running Windows 
98. Most are desktops but several notebooks were purchased for students with space or other necessities. 

One student needed her computer to run the newest version of JAWS for Windows. This entailed a 
memory upgrade and repairs to the modem. This task was contracted to a local computer expert the 
technology specialist had previously contacted and researched. Other equipment needs were Dragon 
Naturally Speaking, and a head pointer for a student with quadriplegia. 

Initial technical training is often spearheaded by individual students’ state vocational rehabilitation 
counselors, but is also supplied as online and telephone desk help from the technology specialist. If 
necessary, locally based tutors are hired to deliver training to the individual. When computers are 
installed in student homes, intensive training is supplied, and lines of communication are established as 
barrier free. During the first three-day face-to-face session, more computer training is supplied and the 
specialist is available before, during and after the sessions for individual tutoring. 

Support beyond the three day face-to-face is via e-mail, telephone help-line and the use of PC Anywhere 
for trouble-shooting the computer itself. The telephone is the main vehicle for support and is often used 
for needs beyond the technical requirements and questions. 

The mode of presentation of the course materials is Courseware version 5 by BlackBoard. Western 
Washington University uses this as its main vehicle for distance education for undergraduate and graduate 
classes. The order of presentation and the ability to contain all learning activities within a web-based shell 
is a definite plus for the pilot program. With the advances in version 5, the accessibility needs of our 
students have been easily met. 

Lastly, focus groups are arranged with each cohort during the second three-day session. The group is 
randomly divided into two groups, and an independent facilitator conducts a one-hour session with each 
group. The results are used for feedback and content analysis for further research. 
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Internet 2 + Web-Based Instruction = 

Effective Distance Teaching and Learning 

Dr. Dixie Mercer and Ms. Frankie Swift 
Project VISION 

Stephen F. Austin State University 

Bacl^round and Need for Training 

Facilitating the education of a student who is blind or visually impaired or one who has multiple 
disabilities including vision loss requires that the practitioner utilize utmost the levels of both the art and 
the science of teaching. Intensive and highly specialized training on such topics as braille literacy, use of 
abacus and Nemeth code, modification of learning materials and environments, orientation and mobility, 
and assessing the needs of students who have multiple impairment is required to equip professionals who 
work with these children. Without the services of a trained Teacher of Students with Visual Impairments 
(TVI) and or Certified Orientation and Mobility Specialist (COMS), students who have visual 
impairments quite often find themselves entering their academic, social, and vocational lives without 
having had an opportunity to master the skills that are specific to their disability and which are essential to 
independent living. In order to help students develop into productive, successful adults frequent and 
intensive service by a TVI and/or COMS is an absolute requirement. Unfortunately, there is a critical 
need for educational personnel who are trained to meet the needs of blind and visually impaired students 
ages 0-22. By the mid 1980s, the TVI and COMS shortage was critical enough to draw attention from 
some of the prominent scholars in the field (Tuttle & Heinze, 1986; Head, 1989). In 2000, the National 
Plan for Training Personnel to Serve Children with Blindness and Low Vision (NPTP) estimated that 
there was a need for an additional 5,000 TVIs and an additional 10,000 COMS. In 1999, the 33 
universities in the United States who have personnel preparation programs in visual impairments 
produced only 215 TVIs and 78 COMS (Com & Ferrell, 2000). 

The Visually Impaired Preparation Program 

Even before the Personnel Preparation Program in Visual Impairment at Stephen F. Austin State 
University (SFASU) was approached by what would become the Northwestern Consortium (Idaho, 
Montana, Washington and Wyoming), it had already acquired experience in developing and 
implementing a distance learning program in collaboration with Texas Tech University. From 1998 to 
2000, these two universities worked in conjunction with Texas School for the Blind and Visually 
Impaired to provide TVI and COMS training across the state. The Visually Impaired Preparation Program 
(VIPP) program utilized a combination of delivery technologies to provide the classes necessary for 
completion of a teaching credential in the field of visual impairment in Texas. Essentially, the classes 
were composed of a once per month meeting over interactive television, followed by independent work 
on the web, and prescheduled computer chat review sessions. Interactive television (ITV) sessions were 
provided on Saturdays so that those who were already teachers (a vast majority of the students) would be 
able to attend. By May of 2000, 194 new prospective VI professionals had registered in the program 
(Griffin-Shirley, Almon, and Kelley, 2000) 

Project VISION — ^Bacl^round 

In the fall of 2000, the VI Personnel Preparation Program at SFASU was contacted by a group of vision 
related professionals in the state of Idaho. This group was interested in getting SFASU to provide distance 
education classes to a consortium of states in the northwest. It was decided that the SFASU would pursue 
a personnel preparation grant through the Office of Special Education Programs (OSEP) in Washington, 
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DC. In the summer of 2001, Project VISION was granted $1.5 million over a five-year period, and 
training actually began in the spring semester of 2002. Twenty-nine students were registered for two 
classes provided by Project VISION. 

Project VISION — ^Program Characteristics 



Philosophy 

Based on their three years of experience in utilizing various distance learning methodologies, the faculty 
at SFASU had become committed to the belief that the curricular content and the learning objective for 
the student should drive the selection of the distance learning technology. Many instructors have tended to 
design their programs around the newest or most popular technology of the moment rather than allowing 
the learning objective to drive the selection of the technological platform (Petrides, 2002; Palloff & Pratt, 
1999; Chute, et. al., 1999; Cyrs, 1997). Further, faculty members agreed that the pedagogical techniques 
for teaching via distance education were (and ought to be) completely different than those which are used 
in a traditional, face-to-face setting. 

Interactive Television (TTV) 

Availability of ITV sites. It was determined before Project VISION was begun that some use of ITV 
would be necessaiy in order to provide students with an opportunity to participate in class discussions, 
instruction/student question answer sessions, and group presentations. Preliminary plans assumed that an 
h.320 compressed video system would be used, but after gathering more precise information, it became 
obvious that these systems would be prohibitively expensive. Further exploration revealed that SFASU 
(and many other major universities) had h.323 (Internet II - 12) capability that might be useable at a 
fraction of the h.320 estimates. In addition to using 12 member sites, it was determined that internet based 
interactive desktop video could be a viable option. Initially, it was thought that there would be very little 
difficulty in finding sites to host the ITV sessions since all that was technically required was an 
interactive video camera and sufficient bandwidth to support video/audio connectivity (presumably a T1 
line). Aside from some manageable problems with server gate keeping systems and occasional bandwidth 
availability, the technology worked veiy well. The major difficulties with the ITV portion of the Project 
revolved around the political and manpower issues surrounding site usage. In spite of these problems, 
however, seven ITV host sites were available by the spring of 2002, and two classes were presented (3 'A 
hours of ITV time each on one Saturday per month). 

ITV pedagogy. By the time Project VISION began, the faculty at SFASU had determined that only those 
activities that required synchronous interaction should be undertaken using ITV. Further, an entirely 
different pedagogy was necessary for facilitating class interaction over ITV. Students had to be made to 
feel comfortable enough to overcome initial reluctance about participating over a televised system. They 
also had to be actively drawn into the class activities so that they would concentrate on the curricular 
content rather than on the technology that was used to deliver it. A light and conversational tone was 
utilized by the instructor, and she made every effort to engage the students on a personal level, attempting 
to make the students feel that they were sitting in a “real” (traditional) classroom. ITV was also used for 
group presentations which utilized Power Point presentations, videotapes, and handouts which were 
downloaded off of the class website. In two cases, these groups were composed of students from two 
different states who were able to use a variety of distance technologies to design and present their 
projects. 

ITV backup. Because the ITV component of the classes was relied heavily on the synchronous 
participation of all students, it was also the delivery mode that produced the highest level of student 
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anxiety when technological problems were encountered.’ It was critical, therefore, that students be 
provided with a reliable back up plan.^ Three systems were put in place to provide more security to 
students who were almost uniformly new to the distance learning process. First, a live, streamed video 
option was made available as a link on the class web site. This site would allow students to watch 
(although not respond or ask questions) the class as it occurred in real time using a Real Media player on 
their home computer. Secondly, a telephone conference bridge was provided so that students could call in 
and participate as the class took place. In one class, a group project was presented over the telephone 
conference bridge. Some students who had a dedicated computer line in their home chose to utilize both 
the video stream and the telephone conference bridge so they could watch as well as verbally participate 
in class.^ Finally, for those who missed the classes altogether, a videotaped version was both available for 
mailing and archived on the web. 

Web Instructional Activities (WIAs) 

Since ITV time was limited as much as possible to class interaction activities, some method of conveying 
basic curricular content to the student had to be devised. The method, Web Instructional Activities 
(WIAs), was developed by the SFASU faculty during the later stages of the VIPP Program. WIAs are the 
rough equivalent of classroom lectures in that they provide the instructor’s personalized approach to the 
material covered and give the student the benefit of the instructor’s unique perspective. Students worked 
on these WIAs asynchronously. 

Learning Activities (LAs) 

In any class, students must be given an opportunity to practically apply the knowledge and skills gained 
through classroom and independent activities. Students in the distance learning program, needed to have 
this same opportunity to interact on a pragmatic basis with the curricular content. This opportunity was 
provided in a set of Learning Activities (LAs) which required the students to apply what they had learned 
in the WIAs. These LAs were posted on the class website, and specific due dates were identified in the 
class calendar (also posted on the website). LAs were completed by the students and emailed as 
attachments to the professor for grading by a student assistant. The LAs were returned to their authors 
with comments, and grades were factored into the student’s semester grade. 

Computer Chats 

Computer chats were used to provide the students with an opportunity to (1) review material for tests, and 
(2) review their answers to test questions. Approximately three to four chats were held during the 
semester in each of the classes. These tended to be very well attended with as many as 1 7 students joining 
in. The instructor had very strict rules about how the chats were conducted, including the following: 

❖ The instructor always began the chat with an overview of the agenda for the session. 

❖ Students were then called on to ask a question in the order in which they had entered the chat 
room. 



’ The computer chats were also synchronous, but these were easily logged for anyone having trouble with live 
participation, and, most importantly, th^ covered no new material. 

^ In point of feet, this happened feirly rarely, although there were two sites (Casper, Wyoming and Helena, 
Montana) that had chronic access and/or bandwidth issues. 

^ There was a 15-20 second lag between the output via telephone and that from the video stream. While this was 
definitely annoying, it made full class possible when winter storms had made driving the hour to class impractical. 
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❖ The instructor would answer the question, open the floor for answers, or refer the question to 
another student. Then she would ask if the questioning student had a follow up. If there was not 
follow up question, the instructor moved to the next student. 

❖ Any student could pass their turn, and since two rounds of questions were usually taken, they 
would typically have another chance to ask questions that had arisen later in the discussion. 

❖ Any student could ask a question at any time by typing a “?” and waiting for the instructor to 
call on them. 



Program Evaluation 

The program was evaluated by the students in a traditional instructional evaluation (dealing with how 
well the instructor had served the learning needs of the students) and also in a technology evaluation 
which attempted to assess how well the technology had served as a platform for learning the course 
content. Students were asked to rate the technological effectiveness in three ways: (1) using a 6 point 
likert scale to rate the effectiveness of different platforms (e.g. ITV, WIAs, LAs, etc.); (2) to rate each of 
the platforms as to whether their use should be increased, decreased, or remain the same; and (3) using a 6 
point likert scale to rate the overall technological effectiveness of the program. A total of 12 evaluations 
were received, and not every student responded to every question. However, results indicated that the 
students were very pleased with the distance education technology and pedagogy used in the class. 72% 
of the respondents rated the ITV sections of the class either “excellent” or “very good.” WIAs and LAs 
were both rated in the top two categories on 90% of the surveys, and computer chats were rated in the 
same categories 80% of the time. The use of discussion boards fell into the top two categories on only 
66% of the evaluations. 

The instructor of both classes also expressed satisfaction with the overall Project performance. In 
particular, the ITV system produced higher quality results with a greater level of reliability than she had 
anticipated before the Project began. 

Project VISION has made it possible to train TVIs and COMS in areas of the United States where such 
training was not a possibility a year ago. In addition, it has demonstrated that expertise can be shared by 
locations that are separated by large geographical distances. It has demonstrated that even very small 
fields (such as visual impairment) can obtain high quality training in a cost effective manner. Hopefully, it 
will also have demonstrated how this process can be continued by geographically contiguous state 
consortia throughout the country which can provide low cost training in working with low incidence 
populations. 
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Designing the Blend for Audiences 
in Developing Countries 

Maureen Miller 
Sr. Projects Officer 
World Bank Institute 

Background 

The International Bank for Reconstruction and Development, frequently called the “World Bank,” was 
established in July 1946. With a goal to reduce poverty and improve living standards by promoting 
sustainable growth and investment in people, the Bank provides loans, technical assistance and policy 
guidance to developing-country members. The World Bank Institute (WBI), a department within the 
World Bank, was established in 1955 to promote development through learning. Its mission is to help 
increase the capacity of countries to make well-informed judgments and carry out effective development 
programs. To this end, WBI facilitates a learning dialogue on development through structured exchanges 
of ideas and experience among people. 

From the beginning, the WBI staff delivered courses in a traditional face-to-face manner to audiences, 
which included high-level government officials and policy makers from developing countries. Course 
teams gathered their participants together in groups of 20-50 for periods of one to two weeks and led them 
through the economic, financial and policy issue based content. In 1995, WBI was reorganized with three 
strategic goals driving the changes: reaching broader audiences, promoting information networks among 
countries and the Bank, and collaborating within the Bank to increase client responsiveness. In late 1997, 
WBI began to look at distance learning strategies as a way to achieve these goals. 

Clients, Challenges, and Initial Solutions 

WBI’s clients come from a broad range of social, economic, educational, and cultural backgrounds. 
Clients (course participants) consist of senior government officials and staff from bilateral and 
multilateral organizations, as well as managers, journalists, NGOs, business and labor leaders, and others 
important to building civic consensus. The main difference between our clients and students in 
universities is that our clients are (often middle age) adults with practical skills in the topics covered by 
the course in which they are going to participate. 

WBI courses are designed to keep pace with its clients’ needs as well as the changing needs in economic 
development. Courses may also be designed for one or more regions with common concerns and may 
consist of multiple activities that span more than one year. This design provides the opportunity for 
participants to gain the broadest possible perspective on specific issues, but it also means that course 
materials are constantly in a state of flux. 

In light of this, the challenges faced in re-designing WBI’s courses for distance deliver, using modem 
technologies, were considerable. In addition to the aforementioned design issues of varied audiences and 
changing needs, the majority of participants had little or no access to computers or the Internet, were 
globally dispersed, and came from a wide variety of cultural and language backgrounds. Noting this, it 
was agreed then that courses must always be learner based. Choosing a common, yet flexible, approach 
to distance education — an approach based upon sound pedagogy and designed to maximize quality 
learning provided by teams of professional specialists using state-of-the-art technology — was paramount. 
Ours would be a learner-centered, integrated, multiple media approach which would, as far as possible, 
replicate the benefits of face-to-face teaching and learning. 
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Taking into account technological infrastructure already available, including the Bank’s worldwide 
network (which connects its country offices and its headquarters) and networks of partner institutions, all 
of which provided some access to videoconferencing, satellite TV broadcasting and Internet, it was 
obvious we could rely on a mbc of technologies — depending on the course objectives, audience and local 
learning sites used. The media mix chosen could be an integrated use of prepared print materials, web 
pages, CD-ROMs, videoconference and live interactive TV, and computer mediated communication (e- 
mail, web and Internet chat). However, after reviewing the various synchronous and asynchronous 
technological tools available, it was decided to rely more heavily on print (for asynchronous activities) 
and one- or two-way video (for synchronous activities). This decision was made for the following 
reasons: a) many courses already had a large amount of background and course material in print; b) it 
was felt that the use of video would provide the “face-to-face” (see and be seen) opportunities that were 
necessary when dealing with key clients — policy makers and high-level government officials. 

Synchronous video access would also provide instructors and subject matter experts the feeling of being 
in a classroom environment, making it somewhat easier for them to accept the distance format. 
Additionally, it was believed that the use of — and access to — computers was limited in client countries, 
inhibiting the effectiveness of their use and the applications they provided. 

Expanding the Toolset and Incorporating Partners 

Today WBI’s mission statement ends with the words . . . investing in people and ideas as the most 
powerful means of development. As a result our participant base has become much more diverse. Now, 
courses must reach people from all walks of life — from farmer to project manager. Minister of Education 
to teacher, women and children in rural villages to high level government officials in capital cities. And 
they must be designed in a fashion in which participants learn from each other, as often the participants 
have as much experience, or more expertise, than do we. 

Changes have been made on the technology side too. Prices for computers have decreased, and the 
wireless technologies are being embraced as a means of quickly “hooking people up.” Due to such 
changes, and the donor community’s pledge that the “digital divide” will not widen, large inroads have 
been made to provide people in developing countries with access to information communications 
technologies, such as cell phones, computers, email and the Internet. 

In the midst of these changes, early in the 2000s, it was conceived that participant reach could be 
expanded by establishing partnerships with other learning networks. And, where those networks didn’t 
exist, client countries could be encouraged to establish Distance Learning Centers (DLCs) with facilities 
providing a gathering place as well as access to videoconference and Internet. Thus was bom the Global 
Development Learning Network (GDLN). The mandate of this global network, composed of public, 
private and non-govemmental organizations, is to work together to build an interactive, multi-channel 
distance learning network for the development goal of fighting to reduce poverty. 

The establishment of this network and DLCs was key to expanding the use of tools and methodologies in 
our course design, particularly in the area of using the newer technologies. When participants can come 
together in a DLC for the learning activities, it is guaranteed that they will have access to high speed 
Internet connection and access to videoconferencing facilities, VCRs, overhead projectors, etc. 
Additionally, the Centers can provide the kinds of logistical, administrative, and technical support that are 
necessary elements of a good learning environment. And by using local facilitators (or facilitating 
institutions) at each location, we have been able to more closely simulate the social aspects of classroom 
learning. 
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Designing the Blend 



As our instructors and subject matter experts move away from being the “sages on the stage” and instead 
become the “guides on the side” and as access to technology increases, designing the blend of tools and 
methodologies for effective learning within our diverse audiences remains a large challenge. Presenting 
course teams with a choice of synchronous and asynchronous technologies including everything - from 
print to videoconference (CD-ROM, videotape, audiotape, Internet, email, videoconference, audio 
conference, TV) gives them flexibility. But assisting them in understanding how (and when) to use these 
tools for effective learning has been the key. 

While there appears to be no typical course (as there is no typical audience and needs continue to change) 
and thus no typical blend, there are some basic elements we have summarized from our more successful 
courses. These include: 

❖ A course can be, on average, 4-8 weeks in length and can reach 20-40 participants in 4-5 
countries. 

❖ A course includes face-to-face discussions (live and virtual), self-paced study (print or web- 
based), group exercises (live and virtual), and both synchronous and asynchronous networking 
opportunities. 

❖ Each site has one or more local facilitators, used much like teaching assistants. 

❖ While this last element was originally designed to provide the instructor with his/her “eyes and 
ears on the ground,” it has had the unexpected outcome of helping to build in-country capacity. 

Following is a brief summary of courses containing these elements. 

Course A 

Audience: High school teachers. Timing challenge: Participants had a full time teaching schedule and 
were only available for synchronous activities during school vacations. Tools: videoconference, 
videotapes, CD-ROMS, Internet, email, print. 

Format: Session 1 - Three eight-hour days together in DLCs with facilitators. Each day consisted of five 
hours local work (including videotaped lectures, group exercises, Internet searches use of videotape, 
group exercises) and three hours group/regional work during which participants in each DLCs were 
connected with each other and the lead instructors in Washington, DC via videoconference. At the end of 
the three days, each participant left with an project and the names and email addresses in his/her assigned 
group. Each group was comprised of six participants and one facilitator. 

Session 2 - Sk weeks, during which participants worked on their assignment both individually and in 
their group via email. 

Session 3 - Two eight-hour days together in DLCs with facilitators during which each individual 
presented their project; and for 3 hours each day, each group presented, via vc, one selected project in the 
group. 

Course B 

Audience: Senior government officials, staff from bilateral and multilateral organizations, members of 
research organizations and academia, journalists. Timing challenge: Participants had full work schedules, 
limiting the amount of time for synchronous activities. Tools: videoconference, Internet, print. 

Format: Six sessions over three weeks. Each session, held in DLCs, consisted of one hour of facilitated 
local group work and three hours of videoconference. Videoconference time was used for input from 
subject matter experts, interactive discussion, and sharing of experiences among the sites. All background 
materials, presentations, participant information, and links to relevant materials were made available on 
the course website. 
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Course C 



Audience: Government officials, private executives, financiers, and academicians. Timing challenge: 
Evening professional development course. Tools: Videoconferences, videotapes, Internet, e-mail, print. 
Format: A ten week program consisting of six videoconference sessions with a variety of subject matter 
experts, and four videotaped sessions shown locally to instigate facilitated group discussion. Participants 
met in the DLCs for each session. Interaction with subject matter experts, following videoconferences, 
was available by email. Course material was made available on the course website. 

Building Communities 

One of the other important things is democracy is not just about voting for your leaders, it ’s about 
intelligent conversation among citizens. And we ’ve lost a lot of that communication in the mass media. 

So . . . will we see intelligent political discourse continue online and will that have an affect? 

(Howard Rheingold) 

We are still struggling with the building and nurturing communities of practice - an objective of the 
majority of our courses. Many of our designs include a threaded discussion space on the course website, 
where this tool is intended to be used to continue the discussions begun by participants and subject matter 
experts during the course. We are finding that some teams do have better results than others with using 
this tool. And it appears that the most critical factor to success is dedicated human resources — 
moderators with the ability to draw participants into the discussion, summarize, and show the appropriate 
online netiquette and personality. 

This may be our biggest challenge yet. 
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A New Typology of Instructional Methods 



Michael Molenda 
Associate Professor 
Indiana University 

Gagne’s Original Construct 

In the very last section of his book. Conditions of Learning, Gagne referred to the different types of 
instructional environments as “modes” of instruction: 

“Environments for learning consist of the various communication media arranged so as to 
perform their several functions by interaction with the student. The particular arrangements these 
media may have in relation to the student are usually called the modes of instruction.” (Gagne, 
1965, p. 285) 

He outlined sfac different modes of instruction being commonly used (Gagne, pp.285-294): 

1 . Tutoring: “interchange between a student and his tutor” 

2. Lecture: “oral communication on the part of the teacher” 

3. Recitation: “teacher ‘heard’ the students perform” 

4. Discussion: “oral communication. . .between teacher and student. . . [and] interactions between 
students” 

5. Laboratory: “a stimulus situation that brings the studait into contact with actual objects and events” 

6. Homework 

❖ self-instruction: “as [reading] a chapter in a textbook” 

❖ practice: “examples of previously learned principles” 

❖ projects: “organize a variety of activities for himself in such a way as to lead to the 
development of a product” 

This construct of “modes” was repeated verbatim in the second edition of Conditions of Learning (Gagne, 
1970), but it disappeared from all subsequent editions. 

Molenda’s Adaptation of Gagne 

It seemed to me that Gagne was onto an important idea here — ^that you could classify the various 
instructional arrangements according to the interaction pattern that characterized it. Gagne did not pursue 
this idea, but I took it one step further, trying to be quite explicit about establishing categories and 
defining the communication patterns among Teacher, Learner, and Resources (Molenda, 1972): 

1 . Tutorial: two-way interchange between tutor (Teacher) and tutee (Learner) 

2. Lecture: one-way information flow from source (Teacher) to many receivers (Learners) 

3. Discussion: two-way interchange among Learners 

4. Laboratory: Learner acts on raw materials (Resources) 

5. Independent study: Learner acts on encoded, instructional materials (Resources) 

6. Practice: Learner uses new skill repeatedly (may be guided by Teacher) 

This seaned to be a handy classification system — small and simple — ^that was fairly successful in 
providing “baskets” into which you could place many of the instructional formats that one encounters in 
everyday teaching (See Table 1). 
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Table 1 . Modes and Formats 



Modes 


Formats 


tutorial 


apprenticeship 

coaching: music lessons 

mentoring 

Socratic dialog 

programmed tutoring 

branching programmed instruction 

adaptive computer-assisted instruction 


lecture 


oral presentation 

overhead transparency presentation 

PowerPoint presentation 

film presentation 

radio program 

television program 

telelecture 




seminar 




T-group 


discussion 


buzz group 
debate 




panel discussion 
role-play 




science experiment 
social simulation 


laboratory 


instructional game 

field work: archeology, anthropology 

case study 

project 


independent study 


reading textbooks, modules, handouts 
reading Web pages 
reading Web “tutorials” 
programmed instruction 
linear computer-assisted instruction 
watching video 




memorization drill 


practice 


language lab 
athletic practice 
drama rehearsal 




end-of-chapter exercises 
recitation 



A theory that accompanies this typology is that what is important is the communication pattern; it 
provides certain possibilities and imposes certain limits. Once you’ve chose the mode of instruction, the 
formats are pretty much interchangeable. The particular manifestation of that communication pattern, the 
format, may bring logistical advantages or disadvantages — ^textbooks might be more readily available 
than Web pages for some audiences — but it’s communication pattern that carries the pedagogical 
power. . . and limitations. We had learned this with a vengeance in the hundreds of media studies 
comparing film and video presentations to live lectures. A given message will have essentially the same 
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learning effect whether it is delivered live, over television, or over 3-D holographs. Choices made for 
logistical considerations — ^time and money — can be crucial decisions, but they don’t directly affect the 
pedagogy. 

Another theory associated with this typology is that certain modes lend themselves to different phases of 
the learning process. For example, a laboratory activity might serve well to stimulate interest and 
provoke questions about a new topic. A lecture would be an efficient way to present new information to 
an audience whose curiosity is aroused. A discussion can help learners digest new principles and apply 
them to their daily lives. Practice activities are often assigned as homework in order to develop 
confidence and speed in application. 



A Transitional Typology 

Over the years, as new technologies and new instructional theories entered the stage the typology grew. 

In 1995, my colleague, Charles Reigeluth, asked permission to put it into print in his book on 
instructional design theory (Reigeluth, 1999, p. 23). See Table 2. 

The New Typology 

Then came Distance Education. When my students and I began to study the pedagogical strategies of 
Web-based distance courses, we quickly found that the 1 999 version of the typology did not provide very 
adequate “baskets” for holding the various activities that were most common in Web-based courses. This 
opened our eyes to the realization that “virtual” courses employ lots and lots of learning strategies, but 
very few teaching strategies. So, in order to capture the teaching and learning methods being employed, 
the typology must be expanded to include both the methods that the Teacher controls and those that the 
Learner controls. This leads to the “new typology.” (See Table 3) 

This typology is intended to portray the somewhat disorderly, asymmetrical picture of reality. The 
categories are not neatly mutually exclusive; there are overlaps. Some constructs could be portrayed as 
separate methods or as subsets of other methods. So, this is today’s view. The next step is to test these 
categories by attempting to apply them to real-world observation of face-to-face and distance education 
courses. Will these “baskets” prove to be comprehensive, distinguishable from each other, and practical? 
That’s the test ahead. 
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Table 2. Typology of Instructional Methods ( 1 999) 


Method 


Visualization 


Strengths 


Lecture/Presentation 




Efficient 

Standardized 

Structured 


Demonstration/Modeling 


(Realistic 

\ / Showing) ' 


Eases comprehension, 
application 


Tutorial 




Customized 
Learner Responsible 


Drill & Practice 


Q © 

(laV^ ^ j 


Automatized 

Mastery 


Independent/Leamer 

Control 


(l)- — C i> *©) 


Flexible implementation 


Discussion, Seminar 




Meaningful, realism, 
owned, customized to 
learner 


Cooperative Group 
Learning 

Games 




Ownership 

Team-building 


Artificial rules 


High Transfer 
High Motivation 


Simulations 




Realistic Structm^ 
Contexr"^-^:r^ 


Discovery 

• Individual 

• Group 


(^T^ (^LA^) 


( LA LA 

^~"'v©y 


Problem Solving/Lab 


oCk Vo 


High Level Thinking in 
ill-structured problems 
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Table 3. Molenda’s Typology of Instructional Methods (2002) 



reacAing Methods ieorning Methods 

(teacher-controUed) (leamer-controUed) 
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How Do Faculty Use Course Management Systems? 
A Modular Approach 



Glenda Morgan 
Learning Technology Analyst 
University of Wisconsin System Administration 

Hal Schlais 

Coordinator of Learning Technology Development 
University of Wisconsin System Administration 



Introduction and Background 

Course ma n age ment systems (CMS) have become both the panacea and the bane of distributed, Internet 
based learning just the past few years. It is the purpose of this presentation to outline three concurrent 
research projects we are undertaking related to course management systems, and provide an update on our 
current activities and proposed directions. 

Distributed learning can be thought of as learning that can be aihanced or accomplished by the delivery 
of learning resources anytime and anywhere. In just a few years the educational commimity has made 
huge financial and pedagogical commitments into distributed learning. Moreover, there has been a sea 
change in attitudes in academia towards the acceptance of Web-based instruction and the use of 
distributed technologies in teaching and learning. The experiment in the use of the World Wide Web that 
began in higher education in the early 1990’s is now seen at every level of education as a potentially 
important pedagogical tool. It has b^me mission-critical to the educational enterprise. 

Like many of our peers, we in the University of Wisconsin System (UWS) have made an extensive 
investment in time, expertise and resources in the enhanc^ent of instruction using these distributed 
learning technologies. The most important among these is the course management system (CMS). Early 
on it became clear that the support and delivery of CMS’s was going to be a substantial commitment. 
Beginning in the fall of 1999 the UWS made a strategic decision, at the behest of its 1 5 institutional 
CIO’s, to support the hosting and delivery of course management systems as a “utility.” Central funding 
was identified, institutions with appropriate technical and support infrastructure agreed to act as “service 
providers” and the earliest vCTsion of our CMS “utility” lTS@Wiscsonsin.edu was bom. Two short years 
later the UWS has over 100,000 of its 150,000 students using these resources each sanester and 
something over 3000 active courses. 

Given the rapid growth and critical nature of CMS’s, providing for them in a cost-effective and reliable 
way has become an increasing challenge for the UWS as it has for many other higher education 
institutions. An important part of this challenge lies in understanding the e-leaming marketplace within 
which CMS’s are developed and provided. Within the UWS the initial strategy was to provide something 
of a research environment as well as to provide diversity and choice among tools. If we look back to the 
beginning of our utility just two and a half years ago, the names of our CMS tools were Web Course in a 
Box (WCB), Learning Space, and WebCT. Of these only WebCT remains. We had barely gotten the 
enterprise underway when Blackboard and purchased WCB. Our most recent attempt at diversification 
was to provide Prometheus, recently purchased by Bb. Clearly change is going to continue to be a 
constant in the marketplace for CMS tools and we need to develop further strategies to deal with it. 

A fiirther challenge lies in finding sufficient funding for course management systems. CMS’s are really 
“academic systems” and increasingly require technical, financial and other academic-specific support a 
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scale previously reserved for large administrative systems like PeopleSoft and Oracle. The central 
funding mechanism is an important strategic choice allowing adoption of these tools independently of 
strapped and already allocated current departmental and college S&E budgets. However, with the CMS 
becoming much more costly we have been forced to rethink our model of how we as support them. We 
have, in fact, questioned whether or not CMS’s are the only way of providing the kinds of support and 
services that faculty and students require. 

As a result, the UWS has undertaken three research projects relating to course management systems. The 
three research projects are as follows: 

1 . First, we undertook a Request for Information (RFI) process in an effort to identify the most robust 
e-Leaming system. 

2. Second we undertook an in-depth project researching just how we use our current CMS tools in the 
UWS. 

3. Finally, we imdertook a series of eight research projects examining modularity, interoperability and 
interchangeability of CMS functionality. 

The RFI Process 

The RFI document itself was issued as a resuh of the work of a UWS task force in May of 2002 with the 
stated purpose of gathering “information about software/services that will meet the Web-based learning 
needs of UWS. The information gathered from this RFI will inform the development of a Request for 
Proposal leading to the acquisition of software/services or set of interoperable software/services to 
support a range of learning needs from blended courses combining online with traditional learning styles 
to frilly web-based asynchronous courses. ... UWS encourages responses that address all or part of the e- 
Leaming system.” 

“The e-Leaming system may be a “best of breed” approach addressing the various functions outlined in 
this RFI. The UWS encourages responses to the RFI from all sectors of the marketplace. Responders 
who cannot provide software/services for the entire spectrum should address the parts that they can 
provide. Responders may offer collaborative responses to address all or parts of the e-Leaming system. 
The results reported on here were summarized by the task force and will be used to generate and RFP for 
the next generation UWS e-leaming system. 

Research on CMS Use 

A second aspect to our research into course management systems consists of an in-depth study of how 
faculty actually use course management systems. Too often it seems that what we know about CMS’s is 
provided either by the vendor or takes the form of a comparison of functionalities of different course 
management systems. We wanted to gain insight into how faculty use course management systems in 
order to get a more nuanced understanding of the following sorts of issues: 

❖ How extensive is faculty use of course management systems? Early findings suggest significant 
under-usage of the tools and our research project will explore in more detail the extent to which 
this is actually the case. 

❖ What features of course management systems do faculty use the most and for what purposes. 
What factors are driving the use of these features? 

❖ Are there differences (beyond a simple difference in degree) in the use of course management 
systems in completely online courses compared to their use in hybrid or blended courses? Is the 
same course management system appropriate for both kinds of courses? 

❖ What are the primary advantages and disadvantages to using a course management system? 
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❖ How adept are faculty and how willing are they to use course management systems in 
conjunction with other tools? 



The insights gained will provide us with the information we need to make decisions about how best to 
provide CMS’s in the future. In order to study the issue we designed a mix of quantitative and qualitative 
research methods. By means of a process of triangulation we were able to capture the multiple aspects of 
faculty use of course management systems. Most of the studies of technology use in teaching and learning 
have tended to be limited to perceptions of the technology or its impacts. In our research project we 
sought to go beyond this. Using a variety of methods we are able to explore not only perceptions of the 
technology and its uses, but also track actual usage within high-end, low and middle of the road users. 

The three parts of the triangulation process consisted of: 

Interviews With Faculty 

Using semi-structured interviews with individual faculty we explored faculty use of course management 
systems by asking 

i. How they came to use the technology, 

ii. How this usage changed over time, 

iii. What features of course management systems they routinely use, for what purpose and with wiiat 
degrees of success, 

iv. Faculty perceptions of the major advantages and disadvantages of course management systems and 
V. Miat other kinds of programs faculty regularly use in conjunction with course management systems. 
These interviews were supplemented with larger focus groups made up of faculty members. 

To get an accurate picture of faculty use of course management systems across different platforms (such 
as WebCT, Blackboard, Prometheus, Lotus learning Space) we interviewed faculty from a range of 
different disciplines and different kinds of institutions (large research institutions, both large and small 
four year comprehensive institutions and two year colleges). 

Survey 

We are carrying out an online survey of University of Wisconsin faculty who use course management 
systems. This survey focused on what features of course management systons they use, for what purposes 
and with what degrees of satisfaction. By means of the survey we hope will get a larger picture of the use 
of the technologies. Surveys will be administered to all faculty currently using a course management 
system within the University of Wisconsin. The site administrator at each institution will be asked to 
forward an online survey url to each faculty member with whom they are currently working. 

Longitudinal Analysis of Usage Logs 

By exploring usage logs of course management system usage over time we are able to get a different 
perspective on their usage by faculty and how this changes. On a number of campuses we have analyzed 
up to four semesters worth of usage logs. By this means we can see how usage changes over time as well 
as getting a broad overview of levels and kinds of usage. This part of the study has needed to be done 
with careful consideration being given to privacy, to ensure that student and faculty privacy are not 
compromised and that data at the level of the individual is not released. 
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Exploring a Modular Approach to CMS Use 



A final part of our research strategy consists of exploring a more modular approach to course 
management use. In this part of our project we are interested in exploring the feasibility of using a range 
of different products alongside and in conjunction with a CMS. From our research into faculty use of 
CMS’s we found that faculty are especially interested in particular functions that CMS’s provide, for 
example, assessment tools, discussion forums and gradebooks however at the same time th^ are 
frequently disappointed by the performance of these tools within the CMS that they currently use. Thus 
we have sought to explore how it might work for faculty to use third-party tools, such as an assessment 
tool, in conjunction with a CMS. We are especially interested in seeing what issues faculty face in using 
third-party tools and how well these will interact with the CMS. To this end we have undertaken a series 
of pilots to explore faculty use of tools. We have pilot studies underway in which we are exploring faculty 
use of third-party: 

❖ Testing and assessment tools 

❖ Discussion and collaboration tools 

❖ Rapid content development tools. 

In addition we hope to add pilots exploring the use of third-party grade books in the near future. 

Conclusion 

By means of these three research undertakings we hope to develop a comprehensive understanding of the 
course management system universe. They have clearly become mission critical tools yet the growth in 
their usage combined with frequent changes in the market and ever-increasing costs mean that steady- 
state is not an option for us. As those responsible within higher education for providing distributed 
learning infrastructure we need to be prepared to meet these challenges and the kind of knowledge that 
our research undertaking are producing is a critical aspect of such preparation. 
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Introduction 

Believe it or not, the use of games that teach can be traced back many thousands of years. For example, 

China used board games and war games over 5000 years ago. Fast-forward a few years ... business games 
starting surfacing in the late 1950s/early 1960s. Even with these examples, the use of games as a learning 
tool has still not gained widespread acceptance. Despite its history, the use of “games” as an instructional 
tool is often viewed with suspicion by serious educators, who wonder, “How can games teach?” “Is this 
just being used for filler, so the teacher doesn’t have to teach?” “We’re not offering coursework to teach 
people how to play games ... we’re running a business here!” 

Times, however, are starting to change. According to Ruben (1999), in the 1970s and early 80s some 
educators embraced games and simulations as “an attractive and novel alternative to traditional classroom 
lectures and other one-way information-dispensing methods” (p. 3). Part of the reason for this, the author 
suggests, is that experience-based (experiential) models have a greater potential to “address many of the 
limitations of the traditional paradigm.” Specifically, they “accommodated more complex and diverse 
approaches to the learning processes and outcomes; allowed for interactivity; promoted collaboration and 
peer learning; allowed for addressing cognitive as well as affective learning issues; and, perhaps most 
important, fostered active learning” (Ruben, 1999, p. 3). 

A 1995 Training and Development survey indicated that 100% of training professionals survey^ indicated 
that thq' used some type(s) of training games and seventy-six percent indicated they used them in all or 
most of their programs (Salopek, 1999). The figures are higher today. Educators in the public sector have 
long used games to enhance the learning process and engage learners, but if our experiences are any guide, 
gaming was typically used in the lower grades — elementary school. All that has changed, as well. 

In a business context, a recent USA Today article discussed the increase in the use of games and toys with 
executive groups as thq^ conduct strategic visioning creativity and planning sessions (Horovitz, 2002). In 
the article, entitled “Toys Bring Out Kid in Corporate America,” the author suggests that cme of the reasoiK 
for the rise in popularity of toys and games is based on the belief that “Most executives are basically kids in 
adult clothing” (Horovitz, 2002, p. 1). The value proposition for many organizations that use gaming and toys 
and that realize the value of “play” is that such things allow us to anchor back to a time when we were most 
creative and open to new ideas — something business and educational environments say they value. 

It is not the purpose of this article to have the final say on the efficacy of gaming in learning. Rather, it is our 
goal to discuss: 

❖ the theoretical foundations behind the use of games in teaching and training in both college and 
corporate settings. 

❖ various uses of games in learning. 
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❖ tools and techniques that help bridge the gap between experiencing the game and tying it to the 
learning objectives. 

❖ the value of games as tools that support different learning styles. 

We have been very encouraged by the learner feedback we have received as a result of our use of games to 
support learning. There is certainly an abundance of literature to support the use of games as tools that help 
learners, among other things, to reflect on new information, reinforce what they already know, enhance 
knowledge transfer, and as a tool for both formative and summative evaluation to support key learning. 

Fundamentals of Games 

What Is a Game? 

There are a number of different terms and definitions used to describe “games” — such as experiential 
learning (Salopdc, 1999), instructional simulations, and group learning (Mantyla, 1999). Carolyn Nilson, 
author of More Team Games for Trainers, defines a game as a “structured activity with learning at the end” 
(Salopek, 1999, p. 29). Steve Sugar, President of The Game Group and author of a book entitled Games that 
Teach, defines games simply as “fim with a purpose” (Salopek, \999, p. 30). The key phrase in this 
definition, in our minds, is with a purpose. At times, we can get more caught up with the “bells and whistles” 
than with the actual content we are trying to transfer to the learner. Fetro and David (2000) are even more 
pointed when they emphasize the importance of course and student objectives when determining how (or if) 
to incorporate educational games into the classroom instruction. 

Why Use Games? 

We consider several factors when deciding whether to use games or not to support learning. 

From a course design standpoint, effective instructional design in both traditional and distance environments 
is driven by how learners are connected to the content, the instructor, and each other. Given this, one of the 
greatest strengths of using games for learning is that they are excellent tools for connecting learners to 
knowledge, key concepts, facts, and processes in a way that is fim and purposeful. Because they are often 
fun, games also help learners construct meaning and to discover things about a subject area in a more 
personal way and in a safe environment. 

In addition to helping learners to “reinforce and review course information by allowing learners to apply 
what they have learned” (Mantyla, 1999, p. 91), “conceptualizing education as a game restores enjoyment, 
healthy competition, cooperation, and discipline to teaching and learning” (Schwartzman, 1997, p. 1). 
Depending on their design, games can also improve teamwork (Sugar, 2002; Nanerow, 1996). 

From a practical standpoint, the profile of leamas has changed (Prensky, 2001). Consida this — ^today's 
children (who are quicl^ becoming tomorrow’s college students and adult leamas): 

❖ spend ova 10,000 hours playing video games. 

❖ receive and said ova 200,000 anails and instant messages 

❖ spend ova 10,000 hours talking on digital cell phones and ova 20,000 hours watching TV (a high 
pacentage fast speed MTV). 

According to Prensky, (2001), they do all of this before they eva even leave for college! As a result, he 
suggests, “learning via digital games is one good way to reach Digital Natives in their ‘native language’ ” 
(Prensky, 2001, p.l). 
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From a technology standpoint, there is a proliferation of new and emerging computer capabilities that enable 
greater collaboration and interactivity across time zones and geography (Berson, 1996). This is not to 
suggest that the traditional barriers to entry relative to technology-enhanced learning do not still surface 
issues such as access, comfort with technology, support infrastructures, etc. It is to say, however, that the 
ability to engage with learners through the use of computers is increasingly attractive, based on leamw 
profiles, and possible, based on current technology capabilities. Prensky (2001) is quick to caution “if some 
games don’t produce learning it is not because they are games, or because the concept of ‘game-based 
learning’ is faulty. It is because those particular games are badly designed” (Prensky, 2001, p. 4). 

From a learner-centered standpoint, games can be intrinsically motivating and can adapt to different styles 
of learners, as well as different learning styles. This is a theme that emerges in much of the literature on 
games (Nemerow, 1997). Adult learning theory suggests that learners like to be in charge of their own 
learning— and they like to experience it. Experience-based learning can foster greater diversity in approaches 
to learning, opportunities for interactivity, collaboration and peer-to-peer learning. If it is true, as some 
suggest, that all learning is social, this is a particularly compelling argument for more active levels of 
learning — something that learning games can satisfy. As well, to be meaningful, the learning venue must 
link to real world problems, ideas, concepts, etc. Games can be constructed to bridge to real-world relevance. 

When constructed with different learning styles in mind, games can often accelerate the learning process. For 
example, because games involve ‘doing,’ they provide the kinesthetic learner with opportunities to engage in 
his or her own learning in a physically interactive way. In an article by Berson (1996), the author, a high 
school social studies teacher, used interactive games with technology as the enabling tool in his class. He 
found that more visual learners were not only able to enhance their critical thinking skills, but they also 
increased their ability to interpret visual cues such as maps and graphs. 

One of the foremost expats in the use of games that teach, Sivasailiam Thiagarajan — ^“Thiagi” — instructs 
trainers and educators that “an awareness of your participants and an undo-standing of their needs and 
learning styles are vital to using training games effectively” (Salopek, 1999, p. 30). This perspective includes 
not only the learning style of the individual, but also the cultural orientation. For example, some cultures are 
less comfortable with games and activities that require high degrees participation in teams or self-disclosure. 

Games can also provide useful tools for generating feedback for the learner as to how well he or she has 
internalized the learning (Sugar, 2002). This feedback loop extends to the instructor, as well. The 
opportunity for both formative and summative evaluation is relevant in both an academic setting, in vdiich 
evaluation is a huge component, and corporate settings, \\4iere the link to business outcomes is of primary 
concern throughout the instructional design process. 



“The commonsense tendency is for people to define play as the opposite of work” (Rieber, 1996, p. 2). 

We have found this statement to be true in many academic and business settings. The mantras, “we don’t pay 
you to play!” and “You’re here to learn, not to have fun!” resonate through the halls of corporate America 
and universities, alike. Why this is so remains one of the ultimate ironies if you consider the positive impact 
of play on things such as stress levels, problem-solving, and creativity. 

Literature on the subject of play (Rieber, 1 998, p. 2) usually revolves around four themes: 

❖ Play as progress — engaging in play leads to other outcomes, such as learning. Play, in this context, 
provides the backdrop for this article. 
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❖ Play as fantasy — play is intended to unleash creativity, which is why things like LEGOs, 

Silly Putty, and Etch-a-Sketch are popping up in corporate America! (Horovitz, 2002). 

❖ Play as self acknowledges that play itself is to be valued without regard to secondary outcomes — 
that is, play for its own sake, to eidiance one’s quality of life. 

Play as power involves our traditional competitive view of play where there are winners and losers. 
In the context of play for learning, however, some authors (Hark, 1997; Nemerow 1996) believe 
that games used to foster learning work best when competitive elements are minimized and 
emphasis is placed on the value of the expaience and learning itself. 

Elements of Games That Teach 

According to Thiagi (1998), well-constructed games share the following characteristics; 

❖ Conflict — Players must have a goal to achieve, as well as various obstacles that must be overcome 
to achieve the goal. 

❖ Control — ^The game must have a clear set of rules that indicate how to play the game (making 
moves, taking turns, etc.). 

❖ Closure — ^It must be clear how and imder wdiat conditions the game ends (e.g., when a certain point 
total is achieved, after a specific time limit, when players are eliminated, etc.) 

❖ Contrivance — ^Those things that allow learners to say, “After all, it’s only a game!” 

❖ Competency — ^Refers to the specific skill or knowledge areas that the game is designed to improve. 

Games and Learning Strategies 

There are several important factors to consider in the instructional design phase of course development — 

where and when to incorporate a game into your lesson. Questions to ask yourself may include: 

❖ Is it important to reinforce specific learning points immediately after a single lesson or should a 
l earning game be incorporated at the end of a unit as a summative evaluation tool? 

❖ Can the game be placed in the beginning of the lesson as a way to create the context for learning 
about a specific topic? 

❖ Is the subject matter highly sensitive (e.g., sexual harassment) and, therefore, requires selecting a 
game that engages the learner but in no way trivializes the subject matter? 

❖ Where can I incorporate my game where it is the most seamless to the learner (doesn’t disrupt the 
flow or instruction or seems as if it was just dumped into the lesson)? 

❖ Are there any logistical considerations that would limit the type of games that are feasible (do I 
have support for any technology-delivered games or must I rely more on a less-sophisticated 
approach when devising and/or selecting a learning game)? 

Other useful questions to consider are offered by Madelyn Callahan (cited in Salopek, 1 999); 

❖ What should the game communicate to the group? 

❖ How large is the group? 

❖ What is the audience profile (demographics, culture, and previous knowledge of the subject matter)? 

❖ Is the game most appropriate for introducing, reinforcing, or demonstrating learning points? 

❖ Is the game “playable”? 

❖ Is a game the best method for achieving my learning objectives? 

Conclusions 

Some of the lessons we have learned in our journey into educational games are that; 
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❖ The use of games as a learning strategy has added great value to the variety and quality of the 
learning experience for our respective audiences. 

❖ Regardless of whether you are in an academic or a corporate setting, games can enhance the 
learning process. 

❖ Every forward-thinking teacher/facilitator needs a techie and every forward-thinking techie needs a 
teacher to partner with when creating games that teach. 

Lastly, we encourage you to visit our web site for gaming ideas and templates: 
http://facstqff. nww. edu/jonesd/games/ 



References 

Berson, M. J. (1996). Effectiveness of computer technology in the social studies: A review of the 
literature. Journal of Research on Computing in Education, 25(4), 486-500. 

Faria, A. J. (1998). Business simulation games: Current usage levels: An update. Simulation and 
Gaming, 29(3), 295-309. 

Fetro, J. V., & Hey, D. (2000). Who wants to be healthy? Journal of School Health, 70(1 0), 417-420. 

Hark, I. R. (1997). It’s how you play the game. Education, 7/5(1), 6-9. 

Horovitz, B. (2002, May 29). Toys bring out kids in corporate America, USA Today, 1-2. 

Mantyla, K. (1999). Interactive distance learning exercises that really work. Alexandria, VA: 

American Society of Training and Development. 

Nemerow, L. G. (1996). Do classroom games improve motivation and learning? Teaching and Change, 3(4) 
356-361. 

Nilson, C. (1998). More team games for trainers. New York: McGraw-Hill. 

Orbanes, P. (2002). Evaything I know about business I learned from monopoly. Harvard Business Review, 
80(3), 51-57. 

Prensky, M. (2001). Digital natives, digital immigrants. Part II: Do they really think differently? [On-line]. 
Available: 

http://www.twitchspeed.com/site/Prensky%2020Digital%20Natives,%20Digital%20Immigrants%20 

-%20Part2.htm 

Rieber, L. P. (1996). Seriously considaing play: Designing intwactive learning environments based on the 
blending of microworlds, simulations, and games. Educational Technology Research & 
Development, 44(2), 43-58. 

Rieber, L. P., & Matzko, M. J. (1999). Serious design for serious play [On-line]. Available: 
http://it.coe.uga.edu/~lrieber/seriousdesign/default.html 

Rieber, L. P., Smith, L., & Noah, D. (1998). The value of serious play. Educational Technology, 38(6), 29- 
37. 



O 



9 



Mungai, Jones, & Wong ❖ 255 



Rubin, B. D. (1999). Simulations, games, and experience-based learning: The quest for a new paradigm for 
teaching and learning. Simulation & Gaming, 30{4), 498-506. 

Salopek, J. J. (1999). Stop playing games. Training and Development, 53{2), 29-38. 

Schwartzman, R. (1997). Gaming serves as a model for improving learning. Education, 118(\), 9-18. 

Sugar, S. (2002). Ten of the very best reasons for using classroom games [On-line]. Available: 
http://www.thegamegroup.com/ 

Thiagarajan, S. (1998). Ask Thiagi. Thiagi Game Letter, 1(4), 6. 

Biographical Sketches 

Dr. Dianne Jones is a professor at the University of Wisconsin-Whitewater. 



Address: 


116 Williams Center 
UW- Whitewater 
Whitewater, Wisconsin 53190 


E-mail: 


jonesd@uw.edu 


URL: 


http://facstaff.uww.edu/jonesd/ 


Games: 


http://facstaff.uww.edu/jonesd/games/ 


Phone: 


262.472.1649 


Fax: 


262.472.3221 



Diana Mungai is founder and Principal of The Melian Group, LLC, a management and organizational 
development consulting firm. 

Address: 7111 Pershing Blvd. 

Kenosha, WI 53142 
E-mail: dmungai@execpc.com 

Phone: 262.697.4001 ; Cell: 262.697.4001 

Loma Wong is Coordinator of the Learning Technology Center at the University of Wisconsin-Whitewater. 



Address: 


Andersen L 1 002 




UW-Whitewater 




Whitewater, Wisconsin 53190 


E-mail: 


wongl@uw.edu 


URL: 


http://www.uww.edu/TIR/its/ltc/ 


Phone: 


262.472.5017 


Fax: 


262.472.5733 




256 ❖ Games to Teach By 



250 



Development of a Software Tool 

for Multi-institutional Faculty with Minimal Computer Experience 

Dr. Deana Namuth 
Distance Education Coordinator 
University of Nebraska - Lincoln 

Ashu Guru 

Computer Software Specialist 
University of Nebraska, DEAL lab 

Background 

In the agronomic and horticultural science areas, emerging research technologies are providing 
opportunities our clientele (adult learners) can utilize to advance their individual careers and businesses. 
This has been especially true with crop genetics and agriculture biotechnology. Often times a challenge lies 
in bridging the gap between the university and its applications beyond the research field or laboratory. 

Our learner backgrounds are diverse, some requiring introductory materials while others needing advanced 
curriculum. FurAermore, their educational goals vary: some earn graduate academic credit, some require 
continued education credits for professional certification, and others want no credit because they are 
participating for their own interests. 

At the same time, advances have also been made in distance education technologies, which have the 
potential to provide an infi’astructure for readily meeting these education and training needs to a larger 
audience. Even though university instructors and extension educators are highly qualified and motivated to 
meet these educational needs, they lack the time and financial resources to also become distance education 
experts or develop complex distance materials. The learning curve for developing distance delivered 
programs can be daunting, especially for already stretched faculty. If a working model for content 
authoring could be established and supported by an electronic environment that was truly user fi-iendly, we 
believed distance delivery of educational materials could effectively bridge the information gap between 
cutting edge agricultural research and its uses by our clientele. 

Objective 

An overall goal of this project was to create both a software program infrastructure and a content authoring 
model for faculty to develop materials within their resource constraints to meet these educational needs 
efficiently. Long term sustainability of both the software and working model was a crucial requirement. 



Project Description 

Land grant universities have a mission to provide unbiased agricultural research, 
college education and public outreach of research information discovered. The 
model of developing entire courses which are password protected was not a 
satisfactory option because this would limit the potential audience, hindering our 
overall mission, as well as place unrealistic expectations upon a single instructor. 
Instead, an idea of an electronic library of lesson materials with free public access 
for educational use was envisioned. With this in mind, a group of faculty could 
individually develop a short lesson and illustrating animation movie addressing a 
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specific content topic in their field of expertise. Collectively these lesson materials would form a resource 
library which public educators worldwide could go to and pull out those particular materials to support 
their specific resident course, distance course, extension program or workshop (Figure 1). These materials 
could also be subjected to a peer review process to insure the learner of accurate up-to-date information 
and provide the authors with a venue for scholarly development and creativity. The electronic learning 
environment would have to be expandable in many aspects: number of lessons per topic, number of topic 
areas addressed, number of content authors who were geographically dispersed, and number of world-wide 
students accessing the site. 

Collaboration Between Information Technologists and Agriculture Scientists 

With this vision in mind, experts at the Distributive Environments for Active Learning Lab (DEAL) at the 
University of Nebraska were consulted. Was there a way to develop a truly user-friendly environment to 
provide the necessary infrastructure? Once fully developed could it essentially stand alone with minimal 
technical maintenance required? What we discovered along the way was the challenge and yet importance 
of clear communication between experts in two very different disciplines of crop science and information 
technology. We found brainstorming sessions effective in which the content authors explained what their 
dream internet library characteristics would be, without considering if the technology was available. It was 
then up to the information technology experts to go back to their labs and see if they could produce such a 
product. It was also helpful for the content authors to look at other case examples to begin developing their 
own ideas. Finally, it became imperative for all involved to keep the larger picture in mind and not merely 
focus on the immediate needs. This was important in order to create an expandable software system and 
authoring model. 



System Description 

The following discussion sections will introduce the features of the developed system in terms of interaction 
of learners with the system and interaction of authors/instructors with the system. We then discuss the 
architecture of the developed system where we briefly describe the tools and technologies that were 
selected; why they were selected and how the developed system allows authors to share resources such as 
images, glossary etc. The content itself remains copyright protected through UNL and the individual 
author’s institution. 



Learner Interactions 

The audiences who wish to view the information 
maintained in this system access the web address 
http://croptechnology.unl.edu (Figure 2). The html 
page at http://croptechnology.unl.edu has a menu bar 
on the left side of the browser window. This menu bar 
contains hyperlinks/buttons that allow a user to make 
selections and browse through the system. For the 
learners the information appears to be organized 
according to individual lessons. A user can select the 
appropriate lesson from either a subject area menu or 
a list of lessons from the left menu bar. Once a lesson 
is selected the user views the first topic of the selected 
lesson. This topic is in form of a HTML page and has 
a next button at the bottom of the page to go to the 
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next topic in this lesson; depending on the topic, a page may also have a previous button at the bottom of it. 
Together, the two buttons allow a user to go forward or backward while reading the lesson content. 

Apart from containing text and graphics, the topic pages have hyperlinked glossary elements. These 
elements can be clicked with the mouse and the glossary word is then displayed to the user in a small pop 
up window. A lesson may also have one or more associated animations, movie clips and/or PowerPoint 
presentations associated with it. The users can access these by clicking on a button on the left menu bar or 
wfren prompted within the lesson’s text. Besides these features, the users who are registered and approved 
can also take a quiz at the end of each lesson. The system maintains the records of each user with 
information on each of the quizzes taken by her/him. A user w^o prefers to read a printed lesson can click 
on the Get Printer Friendly Version button. This presents the contents of the complete lesson in a printer- 
friendly format. 

Author/Instructor Interactions 

According to selected design strategy, the content objects of the system were decided and these included 
topics, glossary, lessons, images, animations etc. with a unique ID identifying each object. These objects 
are stored and managed in a database; when requested by a user these objects are written out in an html 
page format with the help of display managers. 

In order to insert or maintain their content objects, the authors login into their account on the maintenance 
portion of the system. The application program performs the functions of authentication and then provides 
access to the author’s objects. Once logged in, the authors have the capability to add, delete or modify their 
previously maintained objects. The maintenance interface has easy-to-use forms that an author uses to 
insert/edit different types of content objects. The forms interact with their server-side components that in 
turn update the appropriate database tables. Because all the information, including the images and 
animations, are maintained in a relational database and these objects are identifiable by their ID, all these 
resources (including resources from other authors) can be searched and then reused by an author by 
referencing to them through their respective display managers using the object ID. 

System architecture. The complete system operates using Open Source Technologies, namely Apache 
Web Server, MySQL database and application program written in Perl programming language hosted on 
Linux environment. The design and implementation of the developed software is done in a modular way in 
order to provide high flexibility and ease of maintenance. 

A multi-stage approach was taken to analyze and design the infrastructure of the system. The development 
process involved deter minin g the information requirements of the learners and authors. System needs were 
analyzed and structured decisions were made. These decisions included the conditions, alternatives, 
actions, and rules under w^ich the tools will be selected and the software will be developed. Based on the 
evaluated requirements and decisions made, the complete system was emulated with process and data flow 
diagrams. These preliminary models were developed to help make the development process modular and 
manageable. These models formed the skeleton for the system development and implementation cycle. 
Although the models were changed as the system evolved, they provided an indispensable aid throughout 
the development process. 

The decision for opting for the open source technologies was based on the previous experience of the 
development group with the open source tools. The tools that were selected for this system are highly 
flexible, stable, have a wide developer community around the world and are freely distributed. The 
selection decisions were supported by our assessment of the belief such as the Apache being the most 
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popular web server; Perl being the most popular web programming language with its excellent capability of 
text manipulation and rapid development cycle; MySQL being one of the most popular Open Source 
Databases; and the reliability of Linux. Selection of such an operating environment gave us the capability 
to develop a system that is easy to distribute to communities across the world. This could have been 
difficult if proprietary software were used because of the licensing and royalty issues. As a result, the 
content developed for distribution is also free of restrictive licensing. 

Figure 3 illustrates the components of our system that reside on two different servers. The implementation 
is essentially a “fat” server and “thin” client (web browser in our case), where most of the processing is 
performed at the server. The first server is the application server, which holds and executes the application 
code. The second server is the 
MySQL database server that 
provides the database 
management services for the 
information. All the user 
interaction takes place with the 
application server which then 
makes connections to the 
database server for the data 
management/retrieval services. 





Figure 3 System design illustration. 



The Content Authoring Model 

This electronic lesson library was coupled with a content authoring model established at the faculty 
departmental level. In this model, both faculty with teaching appointments and those with extension 
appointments are able to develop distance delivered course modules without being in an overload 
assignment. The idea is that in both cases (teaching and extension), faculty create electronic materials in 
stepwise increments to enhance their current resident teaching or extension programming assignments. Not 
only does this improve the quality of their current activities, but it provides an environment for testing and 
refining the materials, which over time then can be packaged for distance students, as well. 

Further maximizing our resources, we have created what we call dual-purpose modules. These are 
compact, intense, one-credit courses which can be taken for graduate academic credit or professional 
continuing education credit (CEU’s). This shorter length is preferred by the full-time working learners, as 
well as the faculty. In a single offering there will be non-credit learners attending just to gain new 
knowledge, some for CEU credit and some for academic credit. What differs among them is the particular 
assignments and depth of knowledge application required for each group of learners. Currently, this library 
of on-line lessons provides a significant portion of activities students undergo for three of these dual- 
purpose modules. 

Impacts 

What began with one instructor, one graduate student and six crop genetic engineering lesson texts in 
October 1999 has grown to 20 instructors from six universities, three graduate students, two high school 
teachers, seven graphic artists, one programmer, two educational experts, one Spanish translator, one 
professional society and three industry partners to produce a library of 40 lessons covering topics in 
agronomy, weed science, crop genetics, animal science and nutrition and dietetics. More lessons are under 
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development, some in new topic areas, some being modified for a high school audience and others being 
translated for Spanish-speaking learners. 

This library has been utilized in four UNL courses (three distance/grad level; one resident/undergrad) and 
two NMSU (one resident/undergrad; one distance/grad), where in all cases instructors linked to the site 
within other University courseware programs. Animations have been used in several programs at CSU. 
There are many other uses of the library materials, but these mentioned are part of a research study 
underway. More globally, between August 2001 and May 2002 the library has received 260 download 
requests for 5479 flash animation files worldwide to teach a minimum of 5400 learners. Between July 
2001 to March 2002, the site received 809,427 hits from 85 different countries. 

A mirrored site is currently being piloted with the Western Society of Weed Science. This second site will 
host traffic from weed science educators and learners, while periodically being updated with the latest 
content developed at the parent server in Nebraska. 

The advantages of this project we have identified include flexibility of design and layout, ease of sharability 
among authors with varying needs and resources and downloadability of materials for public educators 
worldwide. 
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Building Effective Virtual Teams 
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This paper describes a learning activity that can be used to demonstrate effective selection principles. It 
can also be used to guide students to an understanding of the factors involved in constructing and 
enhancing the performance of a virtual team. 

The paper begins by describing the general principles involved in building an effective team. Various 
task-facilitating roles and relationship-building roles within teams will be described. In addition, 
blocking roles, i.e. behaviors that limit team performance will also be discussed. Differences between 
virtual teams and co-located teams (face-to-face) will be explored 

A multi-stage online exercise is then described. In the first stage, identification of the KSAs (knowledge, 
skills and abilities) of an effective virtual team member is stimulated by a set of suggested discussion 
questions. In the second stage, students are given an opportunity to gain experience in selection by 
developing interview questions related to the KSAs identified in stage one. The third stage of the exercise 
leads students through a process during which they develop performance standards and rules of conduct 
for their virtual team. During the final stage, students generate performance evaluations for each of their 
virtual team mates. 



Introduction 



Qualities of Effective Teams 

The benefits of teams for collaborative learning in the classroom has long been recognized (Bouton & 
Garth, 1983; Jalajas & Sutton, 1984, Johnson & Johnson, 1984; Vella, 1994. Unfortunately, many of us 
have had the unfortunate experience of being on a team that performs poorly or leaves us unsatisfied with 
the quality of interpersonal interaction. Hopefully, most of us have also had the experience of being a 
part of an effective team. What is the behavior tlut distinguishes these two experiences? 

In describing effective team performance within the workplace, Ancona et. al (1999) suggests that there 
are four components: 1) performance - how well do team members produce output in terms such as 
quality, quantity, timeliness, efficiency and iimovation; 2) member satisfaction — how well do team 
members create a positive experience through commitment, trust and meeting individual needs; 3) team 
Ifiaming - how well do team members acquire new skills, perspectives and behaviors as needed by 
changing circumstances; and 4) outsider satisfaction - how well do team members meet the needs of 
outside constituencies such as customers and suppliers? 

These measures of effectiveness can also be applied to teams in a classroom setting. An effective class 
team will work efficiently to provide a product (e.g., research paper, presentation, etc.) that means 
appropriate standards of quality, quantity and timeliness. The team will also be a positive experience for 
its members. Teams members will learn from each other and acquire new skills together and they will 
effectively satisfy the demands of the instructor and peers. 
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Roles Within Teams 



For teams to function effectively, team members must engage in both task and maintenance flmctions 
(Ancona et al., 1999; Gabarro & Harlan, 1983; Schein, 1988.) Task functions help team members to 
organize and focus on getting the work done. Examples include planning agendas for team meetings, 
setting team objectives and prioritizing tasks. Maintenance functions focus on holding the team together. 
Examples include oicouraging all membo^ to participate or mediating conflict between team members. 
Table 1 (extracted from Ancona et al., 1999, M-5, p. 8) provides a list of functions required for effective 
group functioning: 



Table 1. Processes required for effective group functioning 





Process 


Description 


Processes that build 
task accomplishment 


Initiating 

Seeking information 
and opinions 


Stating the goal or problem, making proposals 
about how to work on it, setting time limits. 
Asking team members for specific factual 
information related to the task or problem or for 
their opinions about h. 




Providing information 
and opinions 


Sharing information or opinions related to the task 
or problems. 




Clarifying 


Helping one another understand ideas and 
suggestions that come up in the team. 




Elaborating 


Building on one another’s ideas and suggestions 




Summarizing 


Reviewing the points covered by the group and the 
different ideas states so that decisions can be based 
on full information. 




Consensus Testing 


Periodic testing about whether the group is nearing 
a decision or needs to continue discussion. 


Processes that build 
and maintain a 


Harmonizing 


Mediating conflict between other members, 
reconciling disagreements, relieving tensions. 


group 


Compromising 


Admitting error at times of group conflict. 




Gatekeeping 


Making sure all members have a chance to express 
their ideas and feelings and preventing members 
from being interrupted. 




Encouraging 


Helping a group member make his or her point. 
Establish a climate of acceptance in the group. 




The Virtual Team 


A virtual team shares many characteristics of a co-located team. Like a co-located team, a virtual team is 
a group of people who interact to achieve a common purpose. Unlike a co-located team, however, a 
virtual team does not meet face to face but rather works across space and time to achieve its purpose and 
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relies on communication technologies to achieve the interaction they need. In today’s organizations, 
working together in virtual teams has become quite common. 

Virtual teams, particularly those in the distance education classroom, face some unique challenges in their 
efforts to attain effectiveness. Team interactions frequently occur asynchronously. Thus, the amount of 
time required to thoroughly discuss an issue or reach a group decision is often greatly increased. Perhaps 
an even greater challenge is that there is no opportunity for non-verbal communication. A smile, a blush, 
a look of fimstration can provide important clues to team members about the meaning of messages 
exchanged within a co-located team. Virtual team members are deprived of these clues to their 
teammates opinions, attitudes and emotions (Benson- Armer and Tsun-Yan Hsieh, 1997). 

Thus, while the same types of task and maintenance functions that lead to effective co-located teams also 
need lo occur in virtual teams, accomplishment of these functions can be more difficult to achieve. The 
first stage of the exercise described below encourages studaits to consider the material presented thus far 
and identify KSAs that are particularly important for virtual team members. 

An Online Exercise 

In the paragraphs below, I will describe a multi-stage exercise that I have used in a web based hunm 
resource management class. This class has no face-to-face meetings. The objectives for this exercise are 
to; 

1. provide students with an introduction to the principles of effective selection. 

2. lead students to an understanding of the KSAs that are desirable in an effective team member. 

3. demonstrate the unique challenges inherent in working within a virtual team. 

4. provide students wiA the experience of working with members of a virtual team to accomplish a 
common goal and develop a product. 

5. expose studaits to the steps involved in developing performance standards and operating procedures. 

6. provide students with peer evaluation feedback of their performance within the virtual team. 

Stage One: Identify the KSAs of an Effective Virtual Team Member 

Identifying the specific knowledge, skills and abilities required to fill a position is a critical step in the 
selection of human resources (Gatewood & Field, 1994.) Stage one of this exercise leads students 
through a discussion designed to identify the KSA’s important for a virtual team member. Students 
review an online lecture that summarizes the discussion provided in the first part of this paper with 
respect to qualities and processes of effective teams and the challenges faced by vutual teams. Using a 
web based bulletin board, students are then required to reply to the following discussion questions: 

1. Describe the best experience you ever had working in a team. This could be a team you were part of 
in class, a team you were assigned to at work or a team you were involved with during a recreational 
or volunteer activity. What made this team experience so successful from your perspective? 

2. Describe your worst team experience. Why do you believe this team was not effective or enjoyable? 

3. What qualities are the most desirable for a team member to have? Think in terms of KSAs, i.e. 
specific knowledge, skills or abilities you would like the team member to have. 

4. Consider working with a team in an online environment. How might that experience differ from a 
face-to-face environment? What KSAs would be particularly important for a virtual team member to 
have? (try to list KSAs that haven’t already been identified by another class member so that we can 
generate as broad a list as possible. 

The discussion of KSAs for an effective virtual team member led to identification of the qualities 
presented in table 2 The items with an * were considered particularly important for a virtual team setting. 
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Table 2. KSAs of Effective Virtual Team Members 



• 


listens effectively 


• 


well organized* 


• 


objective 


• 


communicates often 


• 


provides feedback 


• 


respects others 


• 


shows initiative 


• 


flexible 


• 


positive attitude 


• 


trustworthy/ 


• 


dedicated to doing a 


• 


self-motivated 




depfflidable 




good job 






• 


willing to do their fair 


• 


open to the views of 


• 


provides work in a 




share 




others 




timely fashion 


• 


reads and understands 


• 


works well without 


• 


able to compromise/ 




the material 




supervision* 




reach consensus 


• 


communicates 
effectively through 
writing* 











Stage Two: Develop the Interview Question 

In stage two, students were instructed to choose one of the KSAs listed and write 2 interview questions 
that thQ^ believed would help them discern if the person answering possessed the quality they wa-e 
seeking. Students were asked to post each interview question as a separate thread on the discussion board. 

Once all interview questions had been posted, students were instructed to return to the discussion forum 
and post their responses to at least 3 of the interview questions posted by anotha- classmate. Answers 
were to be posted as a reply to the thread started by the question. 

After the deadline for posting interview questions had passed, students were instructed to submit a list of 
preferred team mates. The instructor assigned virtual teams in keeping with the preferences stated by 
students as much as possible. 

A general discussion about the effectiveness of the intaview questions in measuring relevant KSAs was 
held afl^ stage two was complete. During this discussion, a significant level of instructor interaction 
took place in order to guide student understanding of the value of various questions and to provide 
suggestions for ways to better target desired KSAs. 

Stage Three: Develop Performance Standards and Team Policies 

During stage three of the exercise, teams were asked to discuss their expectations for performance. In 
particular, expectations were to be expressed in behavioral terms. Students were also required to develop 
procedures for use if a team member’s performance fell outside of agreed upon expectations. For 
example, extremely high performance might merit an email to the instructor commending the student’s 
performance. Low performance may result in a »nail to the student warning that there peer evaluation 
was at risk if there performance did not improve. Finally, students were instructed to develop a peer 
evaluation form that would be used to provide feedback to the instructor about that student’s performance. 
In addition, each student would receive a summary compiled by the instructor of the comments provided 
by their peers (without any name attribution. 

Stage Four: Perform as a Team and Complete Evaluation of Peers 

During stage four, the virtual team completes a project together. The most recent project I have assigned 
is a case analysis and web-based presentation of the analysis to the other class teams. Each team receives 
a private discussion space to work together on their project. At the conclusion of the project, the team 
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posts a power point presentation, that provides the key points in their analysis for review by the other 
teams. 

Team members are then responsible for completing the performance evaluation form developed during 
stage three. Both quantitative ratings and conunents about each peer team member are required. Upon 
receipt of the evaluations, the instructor compiles an anonymous summary of the written comments 
received to each team member. In addition, the instructor averages the quantitative component of the 
evaluation to determine 10% of the student’s grade in the course (This procedure is clearly announced in 
the syllabus at the beginning of the semester.) 



Conclusion 

This paper describes an online class virtual team exercise grounded in the literature on team effectiveness. 
This exercise provides students with an interactive experiaice in two key human resource management 
activities, selection and evaluation. It also provides them with an opportunity to experience working 
together in a virtual team, a skill highly valued in today’s workplace. In addition, the virtual team 
exercise provides sufficient structure and incentives to engage students and motivate performance. 
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Delivering Interdisciplinary, Competency-Based Professional Education 

On-Line: A Case Example 

Curtis Olson 

Director of Curriculum Development and Evaluation 
University of Wisconsin-Madison Department of Pediatrics 

Rebecca Tribby 

Clinical Nurse Specialist/Case Manager 
University of Wisconsin Children’s Hospital 

Interdisciplinary, Coordinated Care of the Child with Chronic Illness (IC4I) is a graduate-level course 
offered at the University of Wisconsin -Madison to students in pediatrics, nursing, nutrition, social work, 
pharmacy, and child and family studies. Class size ranges from six to fifteen students. Historically, the 
course has been taught in a traditional on-campus format. The faculty have used lecture, group 
discussion, written in-class exercises, role playing, and case simulations to teach students how to function 
as part of an interdisciplinary health care team. 

In the spring of 2000, a decision was made to offer the course in a web-based format in order to reach a 
larger population of health professions students. Implementing this decision posed a number of 
significant challenges: 

❖ The course is taught by as many as 1 0 faculty members represaiting multiple disciplines and 
perspectives on professional education. 

❖ The course would need to be redesigned “from the ground up” to make it suitable for delivery 
via the web. 

♦J» A new consensus and shared vision about the purpose, content, and instructional activities of the 
course would have to be forged. 

❖ Given the caitrality of teaching students in the course how to function more effectively as part 
of an interdisciplinary team, creative strategies would have to be found to promote and facilitate 
teamwork among students enrolled in the on-line version of the course. 

The purpose of this paper is to describe how the course design team dealt with these issues through the 
use of a competency-based approach to instructional design. 

Course Design Strategy 

The most powerful question that can be asked during the design of any educational program is “What are 
we trying to accomplish?” Or perhaps more to the point in professional education, what is it that we want 
students to be able to do in the realm of applied practice? Clear statements of desired outcomes serve as 
guides to subsequent steps in the instructional design process and communicate to learners what they are 
expected to accomplish. To the extent that outcomes describe observable clinical performances, they can 
also help students see the relevance of what they are learning to clinical practice and suggest more 
authentic strategies for evaluating learning. 

For the purpose of developing an on-line version of the IC4I course, a decision was made to use a 
competency-based design strategy. Competency-based approaches to mstructional design are based on 
explicit behavioral specification of both the desired learner outcomes and evaluative procedures for all 
learning activities. Desired outcomes are expressed in the form of competency statements, which 
describe the performances expected of learners if they are to be considered to have successfully 
completed the requirements of the course. 
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To build consensus among the members of the faculty and give direction to the instructional design 
process, we began by developing a detailed set of competency descriptions. In developing competency 
statements, we made a distinction between clinical competencies and course competencies. Clinical 
competencies describe the actual work performance of the professional in a real work setting and 
constitute the ultimate outcome sought by course faculty. Instructional design decisions must be guided 
by this vision of clinical performance to ensure the relevance and effectiveness of the course. However, 
clinical competencies can rarely be observed and assessed directly in the context of an on-line course. 

While they constitute the ultimate desired outcome, they cannot directly serve to guide instructional 
design decisions, hence, the need to define desired outcomes at a second level. Course competencies are 
derived from clinical competencies and describe the performances required of students for successful 
completion of the course. They serve as the basis for decisions about specific learning objectives, the 
course content, learning activities, and evaluation of student learning. While a time-consuming and for 
most faculty, unfamiliar exercise, the process of defining clinical and course competencies is critically 
important, especially in light of the expense of developing on-line courses and society’s (and increasingly, 
accrediting agencies’) demand that professional training programs be able to certify that its learners have 
demonstrated that they are able to perform their professional duties to acceptable standards. 

Example: The Team Function Lesson 

In this section we provide an example of a clinical and course competency drawn from the course’s Team 
Function lesson (one of eleven lessons), describe the process by which they were developed, and 
summarize how they were used to inform choices about course content, structure, learning activities, 
learning resources, and evaluation strategies. 

Competency statements for the IC4I course were developed in a group setting with all faculty 
participating. Faculty met with one of the authors (CO) who served as group facilitator. Working from a 
list of general topics to be covered in the course (i.e., death and dying, team function, teams in health care 
delivery, public policy, advocacy, family perspective, transitions, adherence, cultural competence), the 
group discussed what were the clinical competencies needed for effective practice in each area. Clinical 
competencies were framed without regard to the fact that the course would be taught on-line. Discussions 
typically began with a review of how each topic was currently being taught — ^what content was covered 
and the types of learning activities used. Discussants were encouraged to relate this information to 
clinical practice, primarily in response to two questions: 

Why is it important for students to know/be able to do this? 

How is it useful to them as practicing professionals? 

A total of sixteen clinical competencies were identified for the course using this process. 

The clinical competencies for the Team Function lesson were as follows: 

The clinician will function effectively as a member of an interdisciplinary 
team, be able to monitor the functioning of a team in terms of leadership 

and process, and initiate appropriate interventions to maintain or improve team functioning when 
needed. 

Given a specific case, the clinician will decide on an appropriate approach 
to health care delivery (i.e., interdisciplinary, multidisciplinary, 
trans-disciplinary, or single primary care provider) and choose to use an 
interdisciplinary approach when indicated. 
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In identifying these competencies, the presumption was not that the course would provide all that was 
needed for the student be able to perform them, but instead that they mark out the ultimate goal, and that 
the course would move the student along a continuum in the direction of performance of that goal. Just 
how far along the continuum to that goal students would be expected to progress was an issue that was 
addressed in the next step of the process, when the course competency was defined. 

After developing the clinical competency statements, the faculty were asked to describe what behaviors 
the student would be expected to demonstrate in the context of the course. 

The course competency developed by the faculty for the Team Function lesson was: 

Given a case scenario, the student will be able to interact as part of an interdisciplinary team, 
demonstrating the ability to work collaboratively, communicate effectively, and facilitate 
reaching a consensus. 

The course competency guides the process of formulating more specific learning objectives. The critical 
question is what does the student need to know or be able to do in order to adequately demonstrate 
mastery of the course competency? At this level, much more attention was given to the constraints of an 
on-line method for delivering the course. For the Team Function lesson, the faculty established the 
following learning objectives: 

Describe the characteristics and dynamics of effective teams and barriers to team function 
Describe, analyze, and evaluate the group process of an interdisciplinary team. 

Recognize behaviors of team members that promote or impede team function. 

Use communication skills to enhance team process (c.g., giving feedback, clarifying issues, 
summarizing discussion, testing for consensus). 

Identify and implement appropriate interventions (e.g, agenda setting, conflict management, goal 
setting, negotiation) to maintain or improve group Junctioning. 

In the faculty’s view, if students successfully attained these objectives they would have the many of the 
requisite skills and knowledge to perform the activities described in the course competency. 

The next task was to identify learning activities that would create the conditions needed for students to 
realize the learning objectives. Students would also need opportunities to integrate their new knowl^ge 
and skills and practice the performance required by the course competency statements. Learning activities 
used in the current course and potential on-line activities were listed for consideration. 

The general template for each lesson consisted of an overview page introducing the lessonfaculty and 
describing the topics and learning outcomes (objectives) for the lesson, a learning activities page with 
assigned readings, a lecture (on CD-ROM), a discussion topic, individual and group assignments, and a 
clinical case that required students to apply what they had learned to patient care. 

The learning activities chosen as part of the Team Function lesson were designed around the template and 
included readings on team dynamics and communication; a lecture about different types of teams; 
threaded discussions in which students discussed their personal experience on professional teamis; a brief 
paper critiquing the team function of two videotaped examples of case conferences. In addition, students 
wCTe assigned to teams and asked to complete a group project in which they were to determine the ground 
rules and process that their team would use for future assignments. Three additional group assignments 
during the remainder of the course allowed the students to form their own teams, practice teamwork 
skills, and receive feedback from peers and faculty. 
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Finally, consideration was given to how students’ learning would be assessed. For the Team Function 
course competency and learning objectives, students were evaluated in five ways. First, the faculty 
assessed students’ ability to employ concepts of team functioning presented in lectures and readings to 
reflect on their personal experiences working in teams. Second, they were evaluated on their analyses and 
critiques of videos of pulmonary team meetings. Third, they were evaluated on their ability to function as 
a team member in completing their group assignments. (The course instructor (RT) and faculty were able 
to “eavesdrop” in on the student’s chat and threaded discussions used to complete the group assignments 
throughout the semester.) Lastly, students to provide a self-assessment of the degree to which they had 
accomplished the learning objectives. In the next iteration of the course, additional methods such as skills 
checklists will be used to assess student performance. 

Outcomes 

Through the process described above, the faculty for the IC4I course was able to build consraisus on the 
purpose and general architecture of the on-line course. This process also allowed faculty to describe in 
detail the performances expected of students, identify learning activities that were directly linked to the 
desired outcomes, specify evaluation strategies that would give both the student and faculty feedback on 
course effectiveness, and provide clear guidance to the team that was responsible for developing web 
resources for the course. Piloted in spring of 2002, the course has been highly rated by students, 
recognized on campus as an exemplary on-line course, and shown to be an effective means for 
accomplishing its intended goals. 
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Academic Freedom and Distance Education 
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Abstract 

Faculty and staff at online universities are encountering many of the same political and social concerns as 
faculty in brick-and-mortar environments, but new problems are emerging as well. Online classroom 
activity can be monitored externally in manners not often found in traditional classroom contexts, 
presenting potentials for new forms of academic freedom and intellectual property violations. Participants 
should be vigilant for these violations as they materialize and share relevant accounts with colleagues 
worldwide. 



Introduction 

In past decades, one of the underpinnings of the higher education systems of the US and many Western 
nations has been the notion of “academic freedom” (Hofstader, 1955; Metzger, 1955). Many longstanding 
academic and professional groups have worked to support academic freedom, and new organizations that 
focus on these issues (such as Europe’s Observatory of Fundamental University Values and Rights) are also 
being instituted {Academe, 2002). However, attacks on academic freedom by public officials and 
institutional administrators have grown in recent years, apparently linked to such concerns as international 
security {Lancet, 2002), organizational efficiency (Koch & Fisher, 1998), and campus speech codes (Cox, 
1995), as well as various political pressures. In the advent of the widespread diffusion of online learning 
systems in higher education institutions, new dimensions of academic freedom issues are emerging. These 
encroachments are related to such matters as the control and evaluation of course content and structure. As 
information technology allows the recording and analysis of details of course activities, the classroom (like 
many other workplaces in the information age) can become a “fishbowl” (Clement, 1 992). Distortions of 
various aspects of classroom interaction can be used to harass and intimidate faculty members as well as 
students, providing a chilling effect and leading to diminished innovation and creativity. 

This presentation provides an assortment of scenarios of how distance education is currently influencing 
academic freedom issues in university contexts and how it may affect them in the future. It also outlines 
some of the rhetorical and political battles that are taking place as proponents of distance education counter 
those in academic life who less enthusiastic about information technology advances. The plausibility of 
these scenarios will be supported by details of incidents already reported along with related secondary 
source material. The scenarios reflect themes including the following; 

Online Course Monitoring and Surveillance 

Monitoring of classroom activity may provide frightening restrictions upon freedom of speech and a 
subsequent stifling of intellectual expression. The technology that provides the convenient delivery of 
academic content via computer also affords the surveillance and subsequent analysis of what is conveyed in 
everyday online coursework. Teachers as well as students could fear retribution in years to come if and 
when controversial comments are taken out-of-context. Already, the monitoring of e-mail correspondence 
between teachers and students is causing concern (Alger, 1999). 
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Inability to Reflect Local University Conditions in Course Content 

Many of the nuances of academic life and the various issues of campus may be excluded from 
consideration in course design, thus creating “sterilized and disembodied” institutions delivering 
whitewashed content. Local events in a university context (such as those associated with academic 
politics) often provide a microcosm of the larger issues facing academia and society as a whole, although 
sometimes reflecting conflict as well as collegiality. However, these events may be eliminated from most 
course materials and activities by program administrators eager to produce a “standard” and “safe” 
academic product. The textbook industry in the US (as described in Altbach, Kelly, Petrie, & Weis, 1991) 
may provide a model of the overall academic classroom of the future, as many institutions will focus on 
delivering carefully-scripted educational content that is devoid of most reflections of institutional or 
individual personality. 

“Quality Control” as Mechanism for Control of Content 

Mechanisms for the development and evaluation of online courses may effectively eliminate many varieties 
of academic expression. Quality control (as implemented in “total quality" systems or other 
methodologies) is becoming a growing aspect of universities (Widrick, 2002). Creative nuances in course 
design or implementation that deviate from standard course models may be construed by program 
administrators as technical errors or problems rather than as innovations. For example, faculty members 
who are fhistrated by the course development tools that are sponsored by their universities and who seek 
different channels for course content delivery may be stymied in their efforts. Experimental approaches 
may be eschewed in favor of less creative teaching techniques that are more in keeping with a mass 
audience. 

Restriction of the Use of Certain Sources of Material 

The kinds of sources that are utilized in coursework may be controlled by external bodies, making many 
forms of investigation in classroom contexts impossible or infeasible (Oravec, forthcoming). If certain 
sources cannot be utilized because of economic or political factors then academic freedom can be violated 
in certain contexts. A campus may not be able to afford the intellectual property expenses associated with 
certain materials; this is not enough in itself to claim an academic freedom violation. However, if a faculty 
member’s requests for support are turned down for political reasons, or if access to materials is otherwise 
blocked in a discriminatory manner, a kind of violation of academic freedom occurs. 

Publishing Versus Speech 

The consideration of much of online classroom activity as “publishing” instead of “speech” may remove 
some of the legal and social protections associated wiA classroom speech. Only a small amount of case 
law on these matters has emerged as of yet in academic contexts. 

In general, the trends that the scenarios (along with their theoretical support) reflect show how control of 
the content and modes of delivery of classroom material is being removed from faculty and placed in the 
hands of program administrators. The freedom that individuals have in classroom contexts to speak openly 
as well as to exchange materials may thus be diminished, and university life reduced in quality as a result. 

Rhetorics of Distance Education and Academic Freedom 

As discussed in the presentation, the impetus toward the integration of online or distance education into 
higher education institutions has spawned a rhetoric that champions technology as reflecting “progress” and 
diminishes other aspects of university life (Noble, 2002). This rhetoric includes the following recent 
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statement, in which faculty and their philosophical and political stances have been construed as a barrier to 
the development of online learning: 

It’s not about budgets. It’s not even about network overload. When it comes to applying digital technology 
in the classroom, die biggest obstacle for higher education has nothing to do with resources or wiring. It’s 
about feculty members (Lynch, Altschuler, & McClure, 2002, p. B15). 

Yet another commentator on these issues in The Guardian declares that “the online university has little 
need of faculty in the traditional sense” (Alderman, 2002). With feculty themselves labeled as being 
superfluous to the higher education institution of the future, academic freedom issues are being recast as 
inessential aspects of higher education rather than the very essence of university life; they are the “noise” 
rather than the “signal” of university functioning. 

Some Conclusions and Reflections 

Defending Academic Freedom in Online Contexts 

Rather than being a vestige of the past, academic freedom is becoming even more essential to support open 
inquiry and exchange in an “information age.” Faculty, staff, and students require protection for their 
expressions, especially as the varieties and forms of their inquiry expand with the advanced technologies 
for simulation and discovery at their disposal. However, rhetoric that combines respect for academic 
freedom along with regard for the positive significance of technology is apparently less common than 
confrontational rhetoric that places distance education and academic freedom in opposition. Cases in 
which individual students and faculty members fece severe pressures to conform to values of efficiency in 
course delivery rather than more traditional academic values are also emerging. 

In order to ensure that academic freedom survives in online university contexts, participants must be 
vigilant for academic freedom violations and share their reflections about these matters with each other. 
Unfortunately, with feculty members and students often dispersed across the globe, defense of academic 
freedom cases could suffer. Many of the part-time instructors of distance education courses are 
professionals from non-academic environments and may not be well versed in academic freedom issues. 

Other concerns relate to the lack of cohesion among feculty or even awareness of others’ situations. In 
electronic contexts, casual encounters and exchange of information about university life are indeed possible 
in hallways and coffee room, and allegiances form over committee assignments and golf outings. 

However, if individuals are not aware of the total picture involving the fate of particular feculty members 
they may be less interested in defending those involved. Metzger (1955) described academic institutions 
in flie Western ideal as drawing participants into "working together in the vineyards of knowledge”; as 
these vineyards spread in electronic form across the globe, defense of academic freedom becomes even 
more important. 
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Evaluation in Distance Learning 

With distance learning becoming more prevalent as an option for obtaining an education, many schools 
are rushing to get their courses online and available to students (Institute for Higher Education Policy, 
2000). However, is anyone stopping to ask if what is in place is a quality program? If so, what is the 
definition of quality and what aspects of the program are being evaluated? 

According to the current literature, the majority of articles on the assessment and evaluation of distance 
learning programs focus on single elements and fail to take into consideration the complex environment 
necessary for a quality program. Even titles that convey a holistic approach to evaluation only address a 
specific issue. For example, Benigno & Trentin’s (2000) article, “The Evaluation of Online Courses” 
focuses on assessing student participation through analyzing the students’ messages; Gunawardena, Lowe 
and Carabajal’s (2000) article, “Evaluating Online Learning: Models and Methods” focuses on methods 
for evaluating discussion groups, and Robson’s (2000) article “Evaluating Online Teaching” focuses only 
on the issue of human computer interface. 

Moreover, only a few articles actually provide insight into the complexity involved in evaluating an entire 
distance-learning program. One of the best reports, in regards to explaining the breadth of the issue is 
“Quality on the Line: Benchmarks for Success in Internet-Based Distance Education” by the Institute for 
Higher Education Policy (2000). This report did a thorough review of the literature, selected sfac 
institutions that had reputations as leaders in distance education, and conducted interviews and surveys of 
students, faculty, and administrators at each of the schools. Other articles that provide insight into the 
realm of a holistic program evaluation are Benke, Brigham, Jarmon and Paist (2000) article, “Rethinking 
Quality Assurance: Examining Established Practices, Exploring New Strategies,” which offers valuable 
insight into four of the major elements — faculty, course development, student support, and technology 
considerations — ^and Dasher- Alston’s (1999) article, “Evaluation Criteria for Distance Learning,” which 
focuses on academic planning yet provides additional information on other aspects relating to quality. 

While each of these articles present valuable information into the aspects required in a quality distance 
learning program, none provide all the aspects we, as researchers, feel should be addressed and none 
provide a visual model that illustrated how the elements in question related to one another. Therefore, the 
Concentric Model for the evaluation of quality distance learning programs was created to meet this need 
and is now in the process of being validated. 

Defining a Quality Distance Learning Program 

In order to develop an evaluation model for distance learning programs, a clear definition of what was 
meant by a “quality distance-learning program” had to be established. A survey of people interested in 
the field of distance learning (as determined by membership of listservs dealing with distance learning 
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issues) was conducted to define a quality distance learning program and list the key elements a quality 
program would contain. 

After compiling the results and incorporating elements deemed as critical from the researchers’ point of 
view the following definition was created and used as the foundation of the model; 

A quality distance-learning program focuses on and supports the needs of the people 
it is intended to serve. Therefore, it has at its core the interaction between faculty 
and sttddents, surrounded by pedagogically appropriate content presented through a 
stable technology platform that is supported, both technically and programmatically, 
to provide knowledge and/or training that is accepted and desired by the larger 
community. 

When reviewing the various definitions of a quality distance learning program as provided by the 
participants of the survey, it was interesting to note that the most frequently listed components focused on 
the needs required by the faculty and students of a specific course. Within the various definitions, the 
support needs required to sustain a program were frequently absent. The reason for this can only be 
hypothesized at this point. Was it because the respondents were primarily faculty and were not aware or 
choose not to emphasize the larger picture of supporting an entire program? Is it a weakness in the data 
collection process, in that technical support and administrators did not comprise a large portion of the 
sample? Could it be that the elements outside the everyday classroom are not important in the assessment 
of quality in a distance-learning program? Only further analysis and validation of this definition will tell. 

Development of a Model for Evaluation of Distance Learning Programs 

Using the above definition as the foundation for a model that will assess and evaluate distance learning 
programs, the researchers wanted to capture and build upon their opinion, and that of the participants they 
surveyed, that for a distance learning program to be successful it had to focus and revolve around the 
people it intended to serve - primarily the students who are learning but also the faculty who are teaching. 
In addition, the model needed to position the other elements critical to the success of the program around 
the core of people. 

The model (see figure 1) begins with a core symbolizing the student and faculty member. Each half of 
this core is equally important, as the success of the student learning depends on the success of the teacher 
in providing instruction. Supporting the core is a ring indicating the interaction of the students and 
teachers with the content of the course. The content is surrounded by a course management system, 
which enables the content to be managed by the instructor and served up to the students. We are not 
trying to presuppose that a quality distance learning program needs to have one of the large commercial 
course management systems; we are only suggesting that the content needs to be managed through a 
system for consistency and security of the information. The content, faculty, and students need to be 
supported by a team of technical staff, who are available to assist in troubleshooting problems, providing 
training and instruction, and attending to the server and other technology. This enables individual courses 
and students to be successful, at least in terms of technical expertise. 

This is where the majority of the research ends, in terms of evaluation. However, we suggest that there 
are two additional levels that need to be considered. First, a quality program will need programmatic 
support for the faculty and students. This means that clear support from the executive leadership needs to 
be in place, which will help enable and support faculty needs (e.g. release time for development, 
importance of distance learning to promotion and tenure, etc.) and the needs of the students (e.g. online 
services such as registration and payment, research resources, etc.). Finally, there has to be community 
support for the program. Questions such as is the degree accepted by industry as a valid from of 
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education, can students receive financial aid, and what is the general opinion of the public for online 
education need to be addressed. 




Figure 1: Concentric Model for evaluating distance-learning programs 

Required Elements for Evaluation of a Distance Learning Program 

After the various “rings of support” were established, the researchers reviewed the literature and pulled 
from their own experience to determine the various aspects within each area that should be evaluated. 
This initial analysis yielded 42 critical elements (see Appendix A). These elements were initially 
validated by comparing them to the results provided by the survey participants. Upon review, we noticed 
a large omission from our initial list in that we failed to indicate the importance of addressing ADA 
compliance when developing content. Thus, category 14a was created to address this critical issue. 

Use of the Model 

The model is intended to help involved parties analyze the quality of their program by indicating whether 
each element is something in which they excel, need more work, or are lacking. This will provide an 
indication as to where effort should be placed for improvement, by starting from the inside of the rings 
and working out. Additionally, the model can be used to help establish a distance-learning program by 
using the critical elements to assess the feasibility of success, starting at the outer most rings and working 
their way in. 

Future Plans and Directions 

The next phase in the development process will revolve around the validation of the model through the 
use of expert reviewers and forther review of the literature. During the validation process we will be 
looking to make sure that all the required “rings” of support are indicated and all critical elements are 
defined. If you are interested in participating as an expert reviewer, please contact Elizabeth Osika. 
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Appendix A: Listing of Critical Components in the Concentric Model 



Faculty 

1 . Technical Competency 

2. Knowledge of online pedagogy 

3. Behavioral skill set/motivation 

4. Technology available 

Student 

5. Technical competency 

6. Behavioral skill set / motivation 

7. Technology available 

Content 

8. Interaction between students and students 

9. Interaction built in the class between faculty and students 

10. Design criteria (look, navigation) 

1 1 . Clear objectives and goals (good instructional design) 

12. Assessment / evaluation is consistent with objectives and goals 

13. Learner centered vs. content centered (active classroom) 

14. Does the content make use of the online capabilities 
14a. Adherence to ADA standards for accessibility 

Course Management System 

15. Easy to use for faculty and students 

16. Feature set available 

17. Allows for interactivity 

18. Administrative overhead that is required (money, people, time) 

19. Consistent format 

20. Visually appealing 

Technical Support 
Student 

21. Software support (training on Word, PPT, email, etc.) 

22. Problem resolution - online and in person 
Faculty 

23. Multimedia development assistance 

24. Copyright assistance/policy 

25. Training on course management system and pedagogy 

26. Individual consulting 

27. Problem resolution - online and in person 
Infrastructure 

28. System administration support 

29. Stable, robust infrastructure to support technology 
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Programmatic Support 
Faculty 

30. Commitment from executive leadership/clear direction 

3 1 . Release time for faculty development 

32. Important to promotion and tenure 

33. Peer assistance and mentoring 

34. Policy on Intellectual property issues 
Student 

35. Online registration 

36. Online payment 

37. Online support services (advising, etc.) 

38. All courses (pre-reqs and required) online or transferable 

39. Research resources (library services) 

Community Support 

40. Acceptance of degree by industry (recruitment, job placement) 

41. Financial aid for program 

42. Public opinion / impression of the validity of online services 
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Building and sustaining interactivity are key skills that online instructors need to develop. Although 
many instructors would like to include collaborative learning activities in their online courses, they are 
unsure of how to do so. This is particularly true for those working in the hard sciences and mathematical 
areas. In this paper and in our session, we will provide ideas for including collaborative activity in online 
courses, regardless of the content area being taught. The issue of working with varying learning styles 
will also be addressed. 



The Importance of Collaboration in an Online Course 

Collaboration has often been defined as the “heart and soul” of an online course and that which 
distinguishes online teaching fi-om face-to-face teaching. Collaboration online accomplishes the 
following: 



*> Assists with deeper levels of knowledge generation -r When online classes are developed 
fi-om a constructivist fi-ame, the central theme is the co-construction of knowledge and meaning. 
When working in small groups, teams, or even on the discussion board of an online course, the 
ability to create knowledge and meaning is enhanced. 

♦> Promotes initiative, creativity, and critical thinking - Stephen Brookfield (1987), in his 
discussion of critical thinking notes, “Critical thinking is complex and fi-equently perplexing 
since it requires the suspension of belief and the jettisoning of assumptions previously accepted 
without question. As people strive for clarity in self-understanding, and as they try to change 
aspects of their lives, the opportunity to discuss these activities is enormously helpful (p.lO).” 
Thus, the ability to collaborate enables the development of the ability to think critically, a skill 
that is more difficult to master individually. 

*> Allows students to create a shared goal for learning and forms the foundation of a 

lear ning community - In our previous work, we have noted that a learning community is the 
vehicle through which learning occurs in the online course (Palloff and Pratt, 1999; Palloff and 
Pratt, 2001). Beginning an online course with a discussion of learning objectives and working 
toward a common goal creates not only the foundation of that learning community, but also is 
the first step toward collaboration. If students are clear fi’om the beginning of the course that 
“we’re all in this together,” then incorporating collaborative activity into the course happens 
much more easily. 

❖ Addresses all learning styles - When an online course is developed using learning cycles, 

which are a systematic set of activities that tap into all learning styles, collaborative projects, or 
complex activities, which demand that students approach learning through the use of multiple 
skills, all learning styles are tapped. Consequently, in using collaborative approaches to 
learning, the instructor can be assured that the various learning preferences in the group will be 
utilized and that the less preferred styles may be further developed. 

♦> Addresses issues of culture - Collaborative activity enables the capability of students to 
construct their own knowledge and apply prior experience and their own culturally preferred 
ways of knowing to the task. Consequently, through collaboration, it is likely that a more 
culturally sensitive online classroom can be created. 
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Collaboration, then, assists the instructor and all students in successfully achieving learning objectives 
more easily. Although collaboration takes more time, the outcome is actually a more efficient and 
complete learning process. 

Incorporating Collaborative Activity into an Online Course 

There are numerous ways in which an instructor can create collaboration online, regardless of the content 
area being studied. Some suggestions are: 

❖ Collaborative small group assignments 

♦J* Research assignments which ask students to seek out and present additional resource material to 
their peers 

❖ Group work on case studies 

*X* Simulations 

*X* Shared facilitation 

❖ Homework forums 

♦J* Asynchronous discussion of the reading and discussion questions 

♦J* Papers posted to the course site with mutual feedback provided 

Regardless of how collaborative activity is used online, the instructor is responsible for creating the 
container through which it can happen effectively. Consequently, the instructor needs to set the stage for 
collaborative activity by: 

❖ Focusing on the development of a learning community 

❖ Creating the environment for collaboration by encouraging collaborative activity from the first 
day of the course 

❖ Modeling the process of collaboration through interaction with the group 

❖ Guiding the participants as they engage in collaborative activity, without dominating the 
process 

Additionally, the questions an instructor poses on the discussion board can assist with and promote 
collaboration. Brookfield and Preskill (1999), note that there are several categories of questions that 
promote discussion. They state, “An important focus of democratic discussion should be on getting as 
many people as possible deeply engaged in the conversation. Whatever the teacher says and does should 
facilitate and promote this level of engagement (p.87).” The following are the types of questions they 
delineate that can begin and sustain discussion (pp. 87-92): 

❖ Questions That Ask for More Evidence - These questions are asked when participants state 
an opinion that seems unconnected to what’s already been said or that someone else in the 
group thinks is erroneous, unsupported, or unjustified. The question should be asked as a simple 
request for more information, not as a challenge to the speaker’s intelligence. 

❖ Questions That Ask for Clarification - Clarifying questions give speakers the chance to 
expand on their ideas so that others understand them in the group. They should be an invitation 
to convey one’s meaning in the most complete sense possible. 

❖ Open Questions - Questions that are open-ended, particularly those beginning with how and 
why, are more likely to provoke the students’ thinking and problem-solving abilities and make 
the fullest use of discussion’s potential for expanding intellectual and emotional horizons. 

❖ Linking or Extension Questions - An effective discussion leader tries to create a dialogical 
community in which new insights emerge from prior contributions of group members. Linking 
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or extension questions actively engage students in building on one another s responses to 
questions. 

❖ Hypothetical Questions - Hypothetical questions ask students to consider how changing the 
circumstances of a case might alter the outcome. They require students to draw on then- 
knowledge and experience to come up with plausible scenarios. 

❖ Cause-and-Effect Questions - Questions that provoke students to explore cause-and-effect 
linkages are fundamental to developing critical thought. 

❖ S ummar y and Synthesis Questions - One of the most valuable types of questions that teachers 
can ask invites students to summarize or synthesize what has been thought and said. These 
questions call on participants to identify important ideas and think about them in ways that will 
aid recall. 



How Comfortable Are You in Working With Collaboration? 

An Exercise in Letting Go 

Collaborative activity requires that instructors empower students to take charge of the learning process. 
Consequently, it is important for an instructor, before incorporating collaborative learning into an online 
course to do a self-assessment to determine just how comfortable he or she is with letting go of the 
control. Self-assessment questions for reflection include: 

❖ How do I teach now? What types of learning activities do I include in a face-to-face class? 

❖ How much do I know about small group dynamics? Do I know enough to be able to intervene 
in a group if the process is not going well? 

❖ How will I need to shift or change my teaching style or approach to enable collaboration to 
happen? 

❖ How comfortable do I feel letting go of the control and allowing learners to take charge of the 
process? 

The responses to these questions can determine whether collaborative activity will succeed or fail in an 
online course. The instructor must act as a facilitator or guide, allowing students to create their own 
learning process. It is the outcome of that process that is most critical — how the students get there should 
be of minimal interest to the instructor. 

Evaluating Collaborative Assignments and Collaborative Work 

A major concern expressed by instructors as they embark on collaborative activity online is how to 
evaluate the outcome of that work. Is giving a group grade fair? If not, how does the individual student 
earn a grade? We do use group grading for some of our collaborative projects online. However, we do so 
by asking students to send us a private e-mail giving themselves a grade and evaluating and grading each 
member of the group. We incorporate this into our assessment of their process and outcome. We will 
also use this same process in providing individual grades for a group project. Often, giving a group grade 
for the final product and an individual grade for each group member’s work within the activity provides 
the most equitable means by which collaborative work can be evaluated. 

We also feel that it is critical for students to evaluate the process they went through, which can be done 
publicly on the course site. Finally, regardless of how evaluation occurs, veto power remains with the 
instructor. If we feel that students are being too harsh in their evaluation of themselves and others, we 
will intervene. Concern about how to evaluate collaborative activity should not hinder the inclusion of 
this powerful tool in an online course as it is the most effective means by which to maximize the learning 
process and create a quality learning outcome for all involved. 
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Introduction 

Many online course developers and instructors are committed to providing quality online courses that are 
not only accessible and convenient, but also well prepare learners for the 21^ century as critical thinkers 
and collaborators. Increasingly, constructivism (or the active construction of knowledge by the learner as 
opposed to the passive reception of knowledge from the instructor) is becoming the learning theory of 
choice of course developers and instructors alike to achieve this learning goal. However, available 
research is still limited in terms of providing validated indicators of constructivist principles in online 
courses. 

The purpose of this paper is twofold. First, it describes a study that identifred indicators of constructivist 
principles applied to the development of Internet-based courses; and second, it addresses this study as an 
example of online research — an approach to research that is becoming increasing popular — ^highlighting 
the advantages and disadvantages of the approach taken in this study. 

Indicators of Constructivist Principles in Internet-Based Courses 

This study aimed to assist educators and course developers in providing greater assurance of quality in 
Internet-based courses by identifying indicators that reflect the application of the principles of 
constructivist learning theory to course development. If it can be assumed that online learning is amenable 
to constructivist learning principles and such courses can be markedly improved by adherence to this 
construct, then defining related indicators is important. 

The study used a Delphi survey research method and a panel of experts with identified interests and 
expertise in constructivist learning theory and instructional technology. The panel identified and validated 
important indicators of constructivist principles for use with Internet-based courses. Anonymously 
panelists completed an online Web survey over three rounds of questioning during a one-month period. 
The Web survey automatically updated response frequencies and commentary. In doing so, it permitted 
asynchronous electronic communication between panelists and the researchers. Panelists also 
corresponded with the researchers though e-mail. 

Expert panelists proposed and rated categories and indicators of constructivist-compatible principles, 
which the researchers complied and organized. Over the three rounds, the survey yielded 10 categories 
and a total of 1 10 indicators in 10 categories. Panelists rated 53% of the 1 10 indicators as M = 3.51 or 
higher (on a 5-point scale). A selection of the revised list, with indicators receiving a mean rating of 4.50 
or higher, is presented in Table 1 . The categories and associated indicators provide an initial framework 
that may help educators apply constructivist principles to the development of Internet-based courses. 
However, the researchers caution readers because the category and indicator list does not represent a 
definitive compilation. Additional research is needed to further identify indicators and validate the 
categories and indicators compiled in this study. 
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Table 1 . Indicators Rated 4.50 or Higher 



# 


Category 


Indicators identified by expert panel 


Rnd-1 


Rnd-2 


Ave. 


1 


AOL 


Opportunities to revise or modify work. 


4.50 


5.00 


4.75 


2 


PBL 


Students explore nontrivial problem areas and ask questions, 
debate ideas, make predictions, and draw conclusions while 
creating relevant artifacts. 


4.60 


4.80 


4.70 


3 


COLL 


Examples of collaborative and cooperative small group 
work 

would include: threaded discussions (internal); discussions 
with outside experts; synchronous chats; small-group 
exercises; projects and papers with multiple authors. 


4.60 


4.80 


4.70 


4 


PBL 


Projects are shared with peers during development and 
completion, thus supporting knowledge construction in 
social 

learning settings. 


4.60 


4.75 


4.68 


5 


COG 


Students construct, build, or enact something that is 
representative of an abstract theory or idea. 


4.40 


4.80 


4.60 


6 


AUT 


Example methods of authentic practices include: making or 
building things; analyzing problems; designing solutions; 
trying out solutions; testing and evaluating solutions. 


4.60 


4.60 


4.60 


7 


SAFE 


Instructors avoid public humiliation or unneeded social 
comparison of students. 


4.50 


4.60 


4.55 


8 


SAFE 


High levels of trust and support (which in turn will allow 
greater levels of public criticism) are maintained. 


4.50 


4.60 


4.55 


9 


PBL, 


Project “assignments” are purposefully under-designed to 
allow a large degree of flexibility in interpretation, which is 
in keeping with the construction of new loiowledge. 


4.40 


4.60 


4.50 


10 


COG 


Sample types of tasks or methods include: reflective journal 
entries; threaded discussions; rationales and reflections on 
projects; projects requiring high levels of self-directed work 
and organization; evaluation of others’ work. 


4.20 


4.80 


4.50 


11 


COG 


Students critically evaluate own (and others’) work. 


4.60 


4.40 


4.50 


12 


AUT 


Reported or perceived relevance/utility of the tasks. 


4.40 


4.60 


4.50 


13 


AUT 


Assessments are performance-based or related to real work 
settings. 


4.60 


4.40 


4.50 



Category Code descriptions: 

PBL - Project-based learning tasks 

COLL - Collaborative and cooperative small group work 

COG - Tasks that require higher order cognitive skills 

AUT - Tasks that are authentic, relevant, and meaningful 

SAFE - Safe environment 

AOL - Assessment of learning 

Web Survey Method 

A second aim of the paper is to discuss the use of the Web for data collection and expert panel 
discussions. As mentioned, the study employed a Delphi research method to reach consensus of 
participant opinion through three rounds of questioning. The researchers constructed Web survey forms 
containing tags that interacted with FileMaker Pro database files. From the Web, participants could 
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submit data to and extract them from the database. Interfacing with database files through Web 
documents afforded participants the opportunity to see a compilation of other participants’ submissions, 
and to respond to them in an interactive manner. A database was also useful for data sorting and analysis. 

In all three rounds of questioning, participants submitted their responses through the Web. The first round 
presented four general categories of constructivist-compatible instructional practices obtained from the 
researchers’ review of literature. Using a five-point rating scale, participants rated the importance of each 
category as it related to Internet-based courses. For each category, participants typed at least five relevant 
indicators or statements that describe observable and measurable elements, methods, and/or procedures 
whose presence indicated the use of constructivist principles in an Internet-based course. Participants 
could also type new categories, indicators, and general comments. 

The second round presented the categories of constructivist-compatible instructional practices and several 
indicators (obtained from participants) intended to reflect the application of constructivist learning theory 
to the development of Internet-based courses. For all categories, participants rated the importance of each 
indicator from 1 (Not Important) to 5 (Very Important). The survey also presented the average rating each 
category received in Round 1 . The Web survey tallied responses and compiled comments from the 
previous round. This allowed respondents to consider their responses in relation to those of all other 
participants. 

The third round again presented the categories of constructivist-compatible instructional practices and 
associated indicators. Participants rated the importance of each indicator. To assist participants, the survey 
presented comments about the categories and indicators and the average rating of each indicator from 
Round 2 (see Figure 1). 

Advantage and Disadvantages of the Web Survey Method 

The researchers note three primary advantages about the Web-based survey and online expert panel 
discussions. First, from their perspective, the Web proved to be a viable alternative to expert panels when 
panelists could not meet face-to-face. Web surveys provide unique access to geographically dispersed 
individuals and their interactive capabilities offer much potential for conducting online panel discussions. 

In this case, the Web and database facilities allowed discussants to deliberate about topic categories and 
indicators and to refine them. 

Second, the Web survey was easily accessible to anyone with email and a Web browser. Once the 
researchers created the survey, HTML documents, and database files, they notified potential participants 
through an email that included the site URL. Participants could click the URL and conveniently gain 
access to the survey. The process eliminated the ne^ for a print survey and it having to be distributed and 
returned through the postal system. In addition, the researchers did not need to transfer print copy 
information to digital form and data became immediately available for analysis as soon as respondents 
submitted them. 

Third, compared to print, the Web surveys, as used in this study, enabled interactivity among participants 
and allowed data to be collected dynamically. For example, participants read other participants’ 
comments and saw the mean ratings of categories and indicators, which assisted them in rating and 
refining items. Once participants submitted their responses to the database, their data became instantly 
available for analysis and display. 
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Figure 1; Screen sample of Survey Round 3 



A significant disadvantage of conducting panel discussions in this manner is the limited amount of 
dialogue that took place. The researchers approximate that compared to face-to-face panel discussions, 
limited commentary occurred online. Depending on a survey’s purpose, the utility of the Web for expert 
panel discussions and data access and collection diminishes when participants are disinclined to 
communicate. The amount of commentary that participants contributed across rounds did not exceed 
much more than a thousand words. One may safely assume that if a panel of 5 or more persons met face- 
to-face, the quantity of comments would far exceed this total. The researchers encouraged panelists to 
respond to the comments of others across the survey rounds but limited interaction occurred. From the 
study, it is impossible to ascertain whether the restricted commentary resulted from time constraints (each 
round lasted one week), the researchers not facilitating communications effectively through the Web 
survey interface, or if the panelists simply opted not to respond to others. It is also plausible that since 
computer-mediated (online) groups are task orientated and disinclined to exchange social-emotional 
information relative to face-to-face groups meetings (Chidambaram, 1996; Jonassen & Kwon, 2001) that 
less commentary occurred. The online panelists may have stated their point concisely without engaging in 
extraneous conversations, which may not have been the case if they met in-person. In addition, for many 
individuals, typing text messages into a Web page is more labor intensive and time consuming than 
communicating verbally. Text messages also lack the non-verbal cues that often engage discussants more 
fully in conversation. 

Facilitating Web-base panel discussions online is an area that needs further research. In this study, the 
Web was a useful medium for data collection, analysis, and dissemination. In the researchers’ view, it 
seems reasonable that it can also be as effective medium for panel discussions when interfaces are 
optimally designed and when the panels are appropriately moderated. The researchers did not attempt to 
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moderate the panel. Th^ served only to observe and answer questions when needed. It would be useful 
to understand the degree and type of moderation that fosters increased quality dialogue among 
participants. 



Summary 

In general, the panelists provided much insight about the topic categories and indicators. While their 
dialogue was limited, the researches considered it informative and valuable to the study objectives. The 
results of this work offer a preliminary set of categories with indicators of constructivist-compatible 
principles that may help educators in their Internet-based course development and evaluation efforts. The 
identification of constructivist-compatible principles as they related to Internet-based courses is an 
important area that requires further research and the proliferation of online courses and programs in 
education today emphasizes the need for continued research. 

The Web survey proved useful as an alternative to face-to-face panel discussions. As used in this study, it 
was conveniently accessible to individuals with email and a Web browser. The Web-based survey had 
several advantages over print surveys. Since the data were stored in a database they could be compiled 
and analyzed instantly as participants submitted responses. In the study, there was limited dialogue among 
participants, which may have resulted from the Web interface, among other things. Additional study is 
needed to explore methods that more actively engage participants in discussion. 
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Introduction 

This was an exploratoiy qualitative research project to examine the existence of aggression in the online 
classroom. This particular project was an outgrowth of an earlier focus group >vith online instructors at 
Columbia College where the issue of online aggression had been unexpectedly encountered during an 
enthusiastic discussion of the otherwise positive virtues of online discourse. Almost as an afterthought, 
two instructors in that earlier focus group had this to say: 

Yeah, but there is another side. Students also feel more empowered to attack you online and feel safe doing 
it 

and 

It is safer for students to be negative toward you online because there is not that personal 
relationship. 

Similar results were obtained at SUNY where Smith, Ferguson, and Caris (2001) found that the 
“asynchronous learning environment gives students and teachers time to consider responses to postings, 
but also produces a sense of anonymity and political differences” (para. 1), and that “students are 
sometimes aggressive and questioning of auAority in ways not seen face-to-face” (para. 20). A follow-up 
exploratoiy project to investigate the issue of aggression in the online classroom was initiated. 

On Februaiy 4, 2002, all of the current online instructors at Columbia College {N = 88) were emailed a 
questionnaire asking if they had experienced aggression fi-om students during the course of their duties in 
facilitating an asynchronous course. The instructors were currently involved in teaching an online class (a 
factor which probably limited the response rate), and their experience ranged from teaching their first 
online class, to veteran online instructors who had been teaching online since the first course offerings at 
the institution. Respondents included individuals who were fulltime instructors at the college as well as 
adjuncts. Twenty-five instructors responded (a response rate of 29.5%). A definition of aggression was 
deliberately omitted to allow the response data to operationalize online student aggression rather than 
imposing an arbitrary definition. The general results are summarized in the Table 1 . 

Responses confirmed that aggression in the online classroom was a reality, but an exception rather than 
the rule although three instructors mentioned more than one incident. When it did occur it was 
particularly disconcerting for the instructors, and unprecedented when compared to instructor 
expectations, experience, and training. In their responses most instructors voiced uncCTtainty about how to 
respond as well as shock that the incidents had occurred in the first place. There seemed to be particular 
surprise that their authority had been so actively, and boldly, challenged. 

It is an ironic, or unexpected, outcome of asynchronous online learning that a minority of students 
seems to feel ranpowered to act in a negative or aggressive manner. Ironic outcomes occur when 
emerging technologies have unforeseen effects (Scott, Quinn, Timmerman, & Garrett, 1998). 
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Venues of Online Aggression 



Online aggression was displayed through one or more computer-mediated venues: 

1. Email to the instructor and/or other students. 

2. Conference postings. 

Most of the aggression was directed toward the instructor, but one instructor noted an incident between 
two students. With email the communication tended to be private, but conferences were used for a more 
public forum of expression. Three of the instructors indicated that students had “shouted” (used all capital 
letta's as the cyber version of face-to-face shouting/yelling) during disagreements when it was not their 
typical practice. In descending order of occurrence, the reported instances of aggression were related to 
grade disputes, class expectations/assignments, strong disagreement over controversial topics, or some 
combination. 

Besides shouting, reports included insulting the abilities and intelligence of the instructor (both in 
conferences and email), attempting to take control of conference discussions, and flooding the instructor 
with emails. One instructor noted that the student thqr were having trouble with would send seven or 
more emails asking the same question within an eight-hour period during the day, and each new email 
would be more hostile than the last. One instructor reported receiving a computer virus attached to an 
email that they believed was coimected to a dispute over course assignments. By far the most common 
method of aggressive expression was email or conference communication that included insults, 
insinuation, profanity, demands, threats, etc. Overall the incidents could be characterized as acts of 
disrespect and insubordination. 

Two other areas deserve mention even thoug)i they technically occurred outside of the classroom. Two 
instructors discussed how they had received telqjhone calls at their home at night from argumentative, 
and aggressive, students who were disputing grades. One believed that their caller was intoxicated. The 
other issue is instructor evaluations. Like many other institutions Columbia College uses anonymous 
instructor evaluations for assessment purposes, and has continued the practice within the online 
environment. Instructors believed they could identify the evaluations of students that had been 
aggressive, and that the aggressive students gave particularly negative and vicious evaluations. 

From our findings, including the previous focus group with instructors, it is clear that the perception of 
the online instructors at Columbia College is that the relative anonymity of the asynchronous classroom 
reduces inhibitions, but in some instances results in aggressive behavior beyond anything experienced in 
the face-to-face classroom. 



Theoretical Perspectives 

The data gathered in this study indicated that a single student nearly always instigated aggression alone 
with only one reported instance of two students acting in concert (both were displeased with their grade 
on an assignment). Any theory that successfully predicts/explains the aggressive behavior experienced by 
instructors would have to account for the fact that this is an individual rather than group phenomena. 

The asynchronous online classroom relies exclusively on Computer-Mediated -Communication (CMC). 
CMC provides a starting point for understanding the dynamics of the online classroom, CMC has been a 
subject of research for over 20 years, and the field has produced four major theoretical paradigms “to 
understand which media characteristics of CMC influence users’ interaction patterns” (Kim, para. 1). The 
cues-filtered-out theory is perhaps the most prominent. According to Kim (2000): 
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Its basic argument is that text-based computed-mediated communication lacks physical and social 
cues, which fosters anti-normative and uninhibited behaviors. Bodiless cyberspace has an 
essential shortcoming in which users cannot use gestures, voice tone and facial expressions. In a 
similar vein, CMC lacks shared social norms and standards. It can lead users to be more 
aggressive and impulsive, (para. 3) 

According to this theory CMC filters out the aural and physical cues available in face-to-face 
communication, and increases the likelihood of misunderstandings. The cues-filtered-out theory explains 
and predicts the aggressive behavior of some students. 

Joseph Walther (1996) offered an alternative to the cues-fihered-out theory that “asserts that in CMC 
message senders portray themselves in a socially favorable manner in order to draw the attention of 
message receivers and foster anticipation of future interaction” (Kim, para. 6). Walther’s perspective, 
sometimes referred to as the hyperpersonal interaction model, views the time lag between interaction as 
conducive to thoughtful, and reflective, responses that are superior to face-to-face communication 
because interaction is in essence a “planned discourse” (para. 61). Unfortunately, this perspective does 
little to explain individual behavior that semis to run against group norms. 

The social identity model of deindividuation effects (SIDE) model is perhaps the second most prominent 
model of CMC after the cues-filtered-out approach. “The SIDE model predicts conformity to norms 
associated with the specific social identity of the group, rather than conformity to any general norms” 
(Postmes, Spears, & Lea, 1998). The SIDE model is heavily influenced by “related ... theories of the 
crowd and of mass communication” (Postmes, et al., 1998). The SIDE model emphasizes group 
conformity, and asserts that rather than filtering cues they “can be even stronger in CMC compared to 
face to face, and similarly that power differential, rather than being diluted can be magnified” (Spears, 
Postmes, Lea, & Wolbert, 2002, para. 14). Although the SIDE model does emphasis conformity rather 
than individual deviation from the norm it does raise the interesting question of power differential, and 
the possibility that aggression in CMC might be rooted in issues of power and control. 

Perhaps the most sophisticated theory of CMC is Adaptive Structuration Theory: 

Expanding on the work of Anthony Giddois (1984), Poole, Seibold, and McPhee (1985; 1986) 

formulated a dialectical theory of micro-level social structures This adaptation of structuration 

theory views micro-level structures as being constructed by agents who monitor and control their 
behavior with reference to rules, schemata, logics of actions, and other forms of conventions 
(Donaldson, Scribner, & Perkins, 2001, p. 3). 

“The theory allows for group members to intentionally adapt rules and resources to accomplish goals” 
(Donaldson, et al. p. 3). While the other theories concentrate exclusively on group dynamics to explain 
CMC behavior adaptive structuration theory looks at the complex matrix of individual, social, and group 
behavior in a contextual sense (both group and individual). In the terms of adaptive structuration theory 
aggression by a single student could be explained as that person’s reflective adaptation to the situation in 
an effort to get their needs met. Unfortunately, adapted structuration theory provides little predictive 
power, and might best be described as a meta-theory. 

Adapted structuration theory and cues-filtered-out theory account for aggressive bdiavior by a lone 
student. They are also complimentary. Although there are elements of interest, and merit, in the other 
theories adapted structuration theory and cues-filtered-out theory currently offer the best theoretical 
heuristic for understanding aggression in the online classroom. 
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Discussion 



Using computers to communicate is not a new concept. In 1971 Roy Tomlinson invented a way to move 
email across distributed networks (Zakon, 2002), and in the same year Murray Turoff developed a 
prototype of the computer conferencing system (Rapaport, 1991). Misusing computers, and misusing 
CMC, is not a new concept either. “No matter how technologically advanced or ‘civilized’ we become, 
a gg ression changes its form but is unlikely to disappear” (Garbasz, 1997). That concept was validated in 
the controversial case of the first cyber rape consummated entirely with CMC in the early nineties 
(Dibbell, 1993), and continues with the prospect of anonymity heightening the intensity of classroom 
a ggr ession when it occurs within the asynchronous classroom. Further complicating the issue of 
a ggr ession in CMC is that communication over the Internet has historically been rather anarchistic, 
irreverent, and confrontational (Shea, 1994). 

This was a modest exploratory study, and it did not investigate the student perspective. It is quite 
reasonable to assume that, at least in some instances, the instructor’s behavior either instigated or 
exacerbated a situation. However, it does provide some additional evidence that aggression in the online 
learning environment is indeed an unintend^ consequence of CMC. With the explosive growth of the 
asynchronous learning environment there is a real need for more investigation on what is actually 
happening in the online classrooms, and a need to develop research based policies as well as best practice 
guidelines for instructors. 

One last issue to consider is that online aggression may have little to do with CMC, but may instead be 
related to cmrait troids in higho* education. Instead of aggressive students what we might be seeing is 
aggressive consumers who view higher education as just another market commodity. Worley (2000) 
notes that: 

As learning moves off campus to the home and to the workplace, students will become 
sophisticated consumers in the educational marketplace, expecting the same “services, 
customization, and responsiveness” (Boettcher, 1999, p. 4) that they demand from other 
consumer products, (para. 23) 



Reconunendatioiis 

Tentative recommendations that might reduce the instances and duration of aggressive episodes; 

1 . Clear written guidelines by the institution and the instructor regarding behavior. A written code for 
online behavior that is enforced. 

2. Policies for withdrawing disruptive students. 

3 . Insisting on civility, and staying on topic (issues not personalities). 

4. Dealing quickly with problems, and consuhing with other faculty and staff. 

5. Instructors answering email within 24 hours. 

6. Instructors should not publish home telephone numbers. 

7. Avoidance, or cautious use, of jokes or humor. 

8. Cooling off period. Do not use CMC to respond when upset or angry. 

9. Avoid escalation of the aggression, and realize that aggressive, or seemingly aggressive behavior, 
may be in response to a legitimate student concern. 

10. Actively engage studaits regarding course expectations, the syllabus, etiquette, and policies using 
online quizzes or some other interactive medium. 
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Table 1 . Instances of Aggression Experienced by Instructors at Columbia College 



Experienced Online Aggression No Experience of Aggression Totals 
Number 19 7 26 

Percentage 73% 27% 100% 
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Distance education — ^the ability to reach out beyond the campus to serve the learners at a place and time 
of their choosing — ^has displayed tremendous growth in the 1990s. In 1997-98, almost 44 percent of all 
higher education institutions offered distance-education courses, representing a one-third increase over 
three years (NCES, 1999). Over the same period, these 2-year and 4-year higher education institutions 
also increased their course offerings at a distance by 212% to 54,470 courses (NCES, 1999). 

While the trend toward distance education in adult learning is undeniable, the forces driving this transition 
vary. Many institutions face the dilemma of infrastructure funding constraints while desiring to draw a 
larger market share of the potential student population. Davies (1997) points out that about half of the 
states have projected increased post-secondary education enrollments, but do not have the funding that 
was available to accommodate past generations. Even if this funding was available, Hudspeth and Brey 
(1986, p. 5) poignantly ask one to “imagine the mood of our faculty and students if several thousand more 
tried to find a place to park each week?” 

Distance education offers a potential answer to this dilemma by enabling institutions to be more 
competitive without necessarily requiring a brick-and-mortar commitment. However, distance education 
is a dual-edged sword. While it enables higher education institutions to compete in new arenas, such as 
corporate training, it also adds the burden of increased competition for students from other institutions — 
all while attempting to keep pace with rapid changes in demographics and technology (NCES, 1999; 
Zimmerman, 1995). 

Adult learning institutions are coming to rely more and more on distance education to meet the challenges 
posed by constrained funding, increased competition, and rapid advances in technology. The leaders of 
these institutions are making decisions to move in this direction. However, to be effective, these cannot 
be simple decisions to implement distance education programs. Just as distance education should not be 
considered a panacea or a quick fix, its inception and implementation should not be taken lightly; it 
requires vision, analysis, and coordination to be effective. These same elements are the keys to strategic 
planning. Keller (1983, p. 75) writes that “to have a strategy is to put your own intelligence, foresight, 
and will in charge instead of outside forces or disordered concerns.” Conversely, without a strategy, 
implementation of a distance education program can be reactive; a constant string of incremental changes 
in response to pressure without a clear vision to anchor the process. Such a poorly implemented distance 
education program is not an answer — it is more likely a potential failure and a drain on the institution. 
The key to successful implementation is effective strategic planning. 

Research 

A study designed to develop, refine, and validate a model of the strategic planning process for distance 
education was performed in 2000-2001. To achieve this end, the informed opinion from a population of 
peer-nominated experts was solicited over three rounds of iterative Delphi questionnaires administered 
via electronic mail. In developing this model, the study answered specific questions about the following: 

1. The impact of volatile technological change on a strategic distance-education plan. 

2. The optimum size and composition of a strategic planning body for distance education. 

3. The use of planning assumptions. 
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4. Internal and extmial iactors that are part of the strategic assessment (scan) and analysis for distance 
education. 

5. The development and selection of courses of action in the planning process. 

A straw model, representing a synthesis of the literature on the strategic planning process served as the 
framework for this research. The Delphi panel of experts was asked to identify the issues and questions 
that they believe should be addressed in each of the phases of the straw model. The Delphi questionnaires 
were employed in a sequence that effectively developed, refined, and validated a model of the strategic 
planning process for distance education. 

Impact of Technological Volatility 

The study determined that the volatility of technological change does in fact limit the number of years 
that a strategic plan can project forward; however, comments from the expert panel ^phasized that 
technology does not drive strategic planning. It is the planning process that allows the institution to adapt 
to a dynamic environment by employing an annual review cycle as the proper response to technological 
volatility. Even with annual review, there was strong agreement that projecting 3 to 4 years was the 
optimal target for distance education strategic planning. 

Optimum Size and Composition of a Strategic Planning Body 

Responses from the expert panel indicated that the optimum size and composition of a strategic planning 
body is variable. Ideally, there is a core of 2 to 4 members who jointly design and lead the planning 
process and do all critical writing. This core is joined by a second group, with broad representation of the 
various functional areas and key stakeholders, at key points in the planning process. The panel identified 
17 functional areas that should be considered for inclusion in this second planning group. 

Internal and external &ctors for strategic assessment 

The stucfy identified 216 possible internal and external factors that are part of the strategic assessment 
(scan) and analysis for distance education. After eliminating redundancies, a comprehensive list of 33 
internal and 33 external factors remained. The internal strengths and weaknesses fell into seven areas: 
Institutional Assessment, Leadership, Mission, Stakeholders, Infrastructure, Academic Programs, and 
Funding. External opportunities and threats encompassed eight areas: Market, Competition, 
Customers/Leamers, Politics, Funding, Partnerships, Stakeholders, and Technology. 

P lanning Assumptions 

The expert panel confirmed the role and significance of assumptions in strategic planning. Specifically, 
92% of the panel agreed that all planning assumptions should be documented to ensure that everyone 
involved — even those who come on board later — ^has a common frame of reference. Similarly, over 83% 
of the panel held that planning assumptions are an essential part of a periodic review process. 

Courses of Action 

There was broad agreement (91 .3%) within the expert panel that multiple courses of action (COAs) or 
strategic directions should be developed, analyzed, and presented for a decision. Those who did not agree 
had concerns in two fronts: (1) multiple COAs would only reflect variations in technology, which should 
not drive strategic planning; and (2) leadership intent, time, or infrastructure would in practice limit the 
proposal to a single COA. 
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Strategic Planning Process Model 



la. Task Assignment 

lb. Asset Identification 

l c. Planning Organization 

2a. Leadership Intent 
2b. Planning Schedule 



Planning Initiation 



Planning Guidance/Schedule 



3a. SWOTs analysis 



3b. Needs/Gap analysis 



Analyses 



5a. Logical 
5b. Realistic 

5c. Essential for planning to continue 
6a. Integrate SWOTs analysis 



Mission 4 J 



4a. Vision Statement 
4b. Mission Statement 
4c. Organizational Values & Culture 
4d. Objectives 



Assumptions 4 



6b. Develop tentativeCOAs 
6c. Strategic Alignment 
6d. Refine and expand tentativeCOAs 



Strategy Development/COAs 



7a. Functional Staff Analysis 
7b. Review elanents 
Adequacy 
Feasibility 
Acceptability 
Policy 
Variety 
Completeness 
7c. Decision 



9a Formative Assessment 
9b. Summative Assessment 



Functional Analyses 



Implementation 



8a. Allocate assets 
8b. Detailed plans 
8c. Timetable 
8d. Assign tasks 



Assessment 



Periodic Review 



_ Feedback Loop ^ ^ 

10a. Internal changes 
10b. External dianges 
10c. Assumptions 
lOd. Mission and goals 
lOe. Implementation lesscHis 



The Strategic Planning Process Model includes 202 planning elements and 10 planning phases presented 
in a hypertext format to enable nonlinear navigation. 
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The findings of this study provided a detailed model of the strategic planning process for distance 
education designed to empower higher-education planners to be proactive in the highly dynamic distance 
education environment. An outline of this model is presented in Figure 1 . 
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Program Foundations 

In 1986, Congress mandated joint training for the active components. Title 10, USC (Chapter 38) reads: 

“The Secretary of Defense shall establish personnel policies emphasizing education and experience in 
joint matters for reserve officers not on the active-duty list. Such policies shall, to the extent practicable 
for reserve component, be similar to the policies provided by this [section]” (Section 666, specifics for 
reserve components). 

The FY 99 Defense Authorization Act clarified this guidance: 

“In order to prepare reserve component field grade officers for joint duty assignments, the committee 
directs that a course similar in content to, but not identical to, the in-residence Armed Forces Staff 
College [AFSC] course for field grade active component officers be developed as soon as 
possible. . .periods of in residence training, as well as distance learning, present the best combination of 
academic rigor, cohort development, and cross-service acculturation.” 

In May 2000, the Staff College was tasked to develop a strategy for RC JPME. The requirements 
included achieving learning outcomes comparable to in-residence courses while tailoring the curriculum 
and war-gaming simulations to serve the requirements of 10 separate major commands. In September, 
the final implementation plan went submitted and rejected because of its cost.. 

Planning 

The challenge was to develop a program that achieved learning outcomes in the higher levels of the 
cognitive domain as well as affective learning at a cost that was acceptable. Compounding the problem 
were the facts that the program had marginal support outside of Congress and the faculty team was a team 
of four reserve officers with little education background. The near-term solution was to develop and 
implement a Beta Test to generate support and interest, validate delivery of joint education to reserve 
personnel using advanced distance learning tools, and help to develop the faculty team before a full 
program was attempted. 

Defining the Requirement 

In order to develop a full blended-leaming program similar to the in-residence program, the decision was 
made to test two areas: the ability to deliver joint education at a distance (as described above), and the 
ability to achieve both higher order cognitive learning and affective objectives in this way. Since no such 
course in joint education existed, one needed to be developed. The faculty at JFSC identified a two-day. 
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12-lesson course known as the Joint Planning Orientation Course (JPOC) that was similar in content to 
the full course but not as robust. Since the JPOC existed only in the form of PowerPoint presentations, 
instructors were videotaped delivering the lessons to facilitate their translation into an online format. 

In October 2000, Modem Technologies Corporation (MTC) was contracted to turn this material into an 
online course to serve as the Beta Test. This development proceeded through the fall and into the spring. 



By April 2001 three of the four initial permanent faculty members had reported to JFSC, with the fourth 
reporting in June. From April through June, three of the faculty members participated in faculty training. 
Joint Forces Staff College takes a holistic approach to faculty and course development within each 
resident school or course offered by the college. All faculty members arrive with a baseline of 
knowledge, skills and competencies. New faculty members are highly successful field grade officers, 
service school graduates (Joint Professional Military Education level 1), and hold advanced degrees; most 
of the current faculty members are graduates of the course they will teach. Faculty training and lesson 
development are continuous processes, accomplished through formal training, workshops, individual 
research, a guest lecture program, and visits to the Unified and Specified Commands, Service Component 
Headquarters and Service Staffs, and other governmental and non-governmental agencies. 

Lessom Review 

After establishing the procedures outlined above, only one step remained prior to beginning lesson 
review. The instructors developed a rigorous timeline for course development, identifying the date on 
which each lesson would be delivered to the contractor; the date the lesson would be returned to the 
instructor; the final delivery date of the lesson into the LMS; and the date on which each lesson would go 
“live.” Review of each lesson took place in the following manner; 

1 . Review of the slides online 

2. Correction of spelling, grammar, and format 

3. Reconcile all lessons to existing joint doctrine (to ensure that what was referenced in the lesson was 
the same as what was stated in existing doctrine) 

4. Correction of graphics or identification of new, additional, and replacement graphics 

5. Delivery of corrections/changes to MTC 

6. Receipt of notification from MTC that corrections had been made 

7. Review of corrections by primary instructor and at least one other instructor 

8. Additional corrections delivered to MTC 

9. Finalization of each lesson 

As instructors, the two primary concerns during this review were that each lesson followed the same 
format (Introduction, Content, and Objectives, followed by the course material) and that the lesson 
objectives were written to the appropriate cognitive level. Using Bloom’s Taxonomy for the cognitive 
domain and Krathwohl’s affective domain, the lessons were reviewed to ensure the lesson objectives were 
properly written and that the content matched the objectives. 
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Program Execution 



Test Group Composition 

From July-September, 104 students in four groups (known as cohorts) accomplished the 13 online 
lessons. The initial target student base for the Beta Test was officers of all Services in the grades of 0-4 
(Majors or Lieutenant Commanders) to 0-5 (Lieutenant Colonels or Commanders). Furthermore, the 
desire was to incorporate a balance of job specialties into each of the cohorts to facilitate cross-service 
and cross-specialty acculturation. Although the students who participated in the Beta Test did primarily 
fall into the desired rank categories, there were some who were more junior or more senior. One cohort 
was particular^ senior with a majority of officers in the grade of 0-6 (Colonels or Navy Captains.) In 
another cohort, the job specialty balance was skewed. A majority of officers were in the intelligence 
career field and very few who had operational (field) experience. 

Course Delivery 

These factors notwithstanding, the majority of the student participants were interested and motivated, 
providing valuable input and feedback throughout the course of the Beta Test. Accomplishing two 
lessons every three weeks, participants took part in p^iodic asynchronous threaded discussions intended 
to build on k^ concepts and ideas contained in the lessons. Finally, the students participated in a two- 
day face-to-face exercise, intended to reinforce these k^ concepts and ideas and to allow students to 
work together to solve problems, thus promoting acculturation. 

Program Lessons Learned 



Cognitive 

As discussed in the course development section, significant attention was paid to the lesson objectives. 
Writing objectives at the knowledge and comprdiension levels was quite easy; the challenge came when 
trying to elevate the level of learning beyond that. Early on, the instructors realized that for the purposes 
of the Beta Test it was unreasonable to attempt learning at the synthesis and evaluation level, but 
application and analysis should be achievable. The onfy question that remained was, "how?" 

The first answer came in the form of the asynchronous threaded discussions. Because the interaction 
level of the lessons themselves was low, the ability to achieve application- or analysis-level learning 
within the lesson was curtailed. However, the instructor team realized that by developing complex and 
thoughtful threaded discussion questions these levels could be achieved. This worked for the following 
reasons: 

1 . Students were allowed a relatively limited time period (72 hours) to respond 

2. Students were directed to respond a total of three times, either to the instructor's primary question or 
to a subsequent student response 

3. Instructors performed aggressive redirects or follow-on questions as necessary, to control to the 
extent feasible the threaded discussion objectives 

The instructors found that topics that on the surface appeared simple could be delved for greater depth of 
understanding and deeper levels of cognitive learning. 

The second answer to the question of "how" came in the form of the weekend face-to-face portion. 

During this exercise, students were presented with a scenario designed to allow them to solve problems 
using tools learned during the online lessons. Information originally presented in the knowledge and 
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comprehension levels quickly melded to raise levels of learning to application and, in some cases, 
analysis. 

Affective 

The challenges associated with affective objectives are many, not the least of which is successfully 
measuring them. During the Beta Test, affective objectives were achieved primarily during asynchronous 
threaded discussions and the face-to-face sessions, with the face-to-face sessions allowing for the 
culmination of ideas learned and discussed throughout the course of the Beta Test. It is important to not 
here that the affective objectives in this course were most impacted by the blended- learning delivery 
method, because had the course been online only any affective learning would have remained at the lower 
levels. Having the two-day face-to-face portion enabled the deepening and broadening of affective 
learning. 
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Background 

The final year of the Doctor of Pharmacy program at the University of Nebraska Medical Carter (UNMC) 
College of Pharmacy (COP) consists of experiential training in a variety of health-care settings. This 
experiential training is organized in 4 week blocks, called clerkships; students complete 10 clerkships in 
their final year of the Pharm.D. program. During the clerkships, students are precepted by full-time 
faculty membCTS of the COP and by adjunct faculty (on-site pharmacists). 

In the Geriatric Pharmacy Clerkship, students work with pharmacists in long-term care nursing facilities 
and short-term care rehabilitation facilities. This on-site clerkship training is supplemented with learning 
activities and face-to-face wedtly conferences conducted by a full-time faculty member. The weekly 
conferences involve student analyses of a series of patient case scenarios having medication-related 
problems. Students develop and present patient-specific treatment plans that are discussed in conference, 
and they act as peer-reviewers. The faculty member facilitates discussion, probes the students’ decision- 
making processes, seeks clarification and elaboration of the students’ recommendations and rationale, and 
asks students to respond to variations of in the patient case scenarios. Student performance in the weekly 
conferences is evaluated. 

In Spring 2001 the COP faculty member precepting the Geriatric Pharmacy Clerkship started a process of 
shifting the face-to-face weekly conferences to an online asynchronous group discussion using 
Blackboard courseware. Currently, approximately 25 percent of the group learning activities in the 
Geriatric Pharmacy Cl^kship are conducted online. TTiis clerkship has bwome an example of a blended 
course (clerkship) in which online case-based learning activities link to curricular and institutional goals. 

Rationale for a Blended Coarse 

Distance education methods are used in the Geriatric Pharmacy Clerkship for several reasons. Typically 
there are 5-10 students in the clerkship at the same time. Student performance in the weekly discussions 
of patient case scenarios is evaluated. Given the size of the groups and the complexity of the discussions, 
it has been a challenge to complete a consistent and rigorous evaluation of each student. The permanent 
written record created by an asynchronous online discussion provides a format for more deliberate 
assessments of student performance in a discussion and decreases the difficulty encountered when trying 
to evaluate student performance “on the fly.” 

The on-site training in the Geriatric Pharmacy Clerkship occurs in a variety of geriatrics-related health 
care settings in the Omaha, Lincoln, and Fremont, Nebraska area. All sites are within an hour’s drive 
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from the College of Pharmacy, located in Omaha on the UNMC campus, where are the face-to-face 
weekly discussions are held. Most students have Internet access at their clerkship sites or at home in 
addition to access from computer clusters at UNMC. The online discussion format has the potential to 
allow students to spend more time with their on-site pharmacist preceptors and spend less time traveling 
between their clerl^hip sites and UNMC. In addition, and perhaps more significantly, the online 
discussion venue facilitates the development of distant clerkship training sites, sites that previously would 
not have been considered because of the long driving distance to UNMC to attend the faco-to-face weekly 
discussions. 

Incremental Migration to the Online Format 

An incremental migration to the online format was chosen because most students beginning their 
Geriatric Pharmacy Clerkship limited experiaice with online asynchronous discussions. In addition, it 
was important to model the format of and expectations for the patient case discussions in a face-to-face 
conference prior to starting an online discussion. Lastly, an incremental approach was chos^ in order to 
generate experience with asynchronous discussions in this clerkship setting and incorporate student 
feedback b^ore shifting a larger portion of the patient case discussions to the online format. 

Curricalar Goals and Strategic Outcomes 

Educational outcomes for the UNMC Doctor of Pharmacy program include general ability outcomes and 
professional practice ability outcomes. There are several general ability outcomes supported by online 
asynchronous active learning. Some of these outcomes pertain to student use of appropriate technologies 
in critical thinking processes, communication abilities, and self-learning abilities. Several of the 
professional practice ability outcomes enhanced by online active learning activities in the Geriatrics 
P harma cy Clerkship. These outcomes pertain to student abilities in the use of technology to appropriately 
research and develop patient-specific treatment recommendations and to respond to questions about the 
effects and appropriate use of medications. 

Part of the mission of the UNMC COP is to train pharmacy practitioners who meet the pharmaceutical 
care needs of the people of Nd>raska, including those in under-served areas of the state. Educational 
models for experiential training in rural and underserved areas of Nebraska may be enhanced by the 
ability of &cuhy members to engage distant students in discussion of patient cases and medication-related 
problans. 



Evaluation Components 



Evaluation of Student Performance 

The implementation of online learning activities in the Geriatric Pharmacy Clerkship has led to a 
reexamination of the assessment methods used to evaluate student performance in the discussion and 
analyses of patient case scenarios. A criterion-referenced assessment tool for use in this setting has been 
developed to assess the appropriateness and clarity of student recommendations and student peer review. 
The standards for performance are discussed with students during the orientation to the clerkship. These 
standards, along with an explanation, are delineated below. 

1 . Recommendations for treatment and monitorii^ must be specific. Each recommendation for 
treatment and monitoring (i.e., evaluation of treatment outcome) must be specific enough so that the 
appropriate health care professional could complete and implement the recommendations as they 
were intended. 
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2. Recommendations for treatment and monitoring must be complete. All essential treatments and 
monitoring parameters must be addressed. All essential elements for each recommendation must be 
present. 

3. Recommendations for treatment and monitoring must be appropriate. Each recommendation 
for treatment and monitoring must be consistent with current standards of practice and must be 
tailored to the specific health needs of the patient. 

4. Recommendations for treatment and monitoring must be appropriately justified. The rationale 
must include a defense of the treatments that were recommended and an explanation of why the 
major alternative treatments were not recommended. The rationale must include an explanation of 
how each recommendation is tailored to the specific health needs of the patient. 

Student Evaluation of Face-to-Face and Online Conferences 

Since Spring 2001, a total of 40 students have completed the Geriatric Pharmacy Clerkship over the span 
of 7 four-week clerkships. These students were debriefed following the completion of the online learning 
activities. A consistent set of questions was used during the debriefing sessions. 

Students frequently reported several positive aspects about the online discussion. (1) They could reflect 
on the discussion, consult resources (including online resources), and thoughtfully prepare their 
responses. (2) They could manage their schedules and participate in the online discussion at then- 
convenience (within constraints set by the instructor). Also, some students said that the online discussion 
was “more open,” that it was easier for them to state their disagreement with some of their peers’ 
recommendations in the online discussion compared to the face-to-face setting. 

Students disliked several aspects of the online discussion. (1) In comparison with the face-to-face 
discussions, students were dissatisfied with the lack of immediate feedback to messages they posted 
online. The typical comment in this regard was “I did not know if the discussion was going in the right 
direction.” (2) Students disliked having to take the time to reread messages in order to review the context 
for new messages posted to the online discussion. 

When asked how the online discussion could be improved, students related their lack of experience with 
online threaded discussions to their challenges in navigating through an online discussion of multiple 
treatment issues. They identified the importance of appropriately posting related messages in a threaded 
discussion and the value of using the message subject line to give the clear indication about the content of 
a message. In response to this, a practice online discussion was implemented, prior to the beginning of the 
online patient case discussion, using an exercise appropriately named “Two Truths and a Lie.” In this 
exercise each student started a new thread by posted a message containing three statements about him or 
herself; two of the statements were true, and one was a lie. Then each student would post a reply in the 
threads started by other students and attempt to identify the lie about that student. This activity, which 
turned out to be a fun and, at times, entertaining activity for students, helped students improve their ability 
to organize messages in a threaded discussion and to navigate through the messages. 

Students were asked if they felt that they learned more in a face-to-face discussion or in an online 
discussion. The majority of students indicated that thQ' felt they learned more in a face-to-face 
discussion. Some students said that they learned more by listening to a discussion than reading online 
message. Others indicated that nonverbal cues from the instructor and peers were important to their 
learning. Yet, when asked if they felt that the quality of their responses was better in a face-to-face 
discussion or in an online discussion, many students felt that the quality of their online responses was 
better (because they could take time to consult resources and draft responses). In trying to clarify this. 
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some students said that they prepared more broadly in advance of face-to-face meetings while their 
learning was more in-depth, or focused, in the online discussion. Perhaps 1 in 5 students related their 
better quality of online responses to a perception that they learned more in the online discussion than a 
face-to-face discussion. 



Lessons Learned 

1 . The use of an online discussion “icebreaker” helped students develop skills using the asynchronous 
discussion technology and helped build community. 

2. Many students felt that they learned more broadly in the face-to-face discussions compared to the 
online discussion. They felt that their learning was more in-depth during the online discussion 
compared to the face-to-face discussions. 

3. A criterion-referenced assessment tool was a practical and efficient means of assessing student 
recommendations in face-to-face and online discussions. 

Next Steps 

1 . The criterion-referenced assessment tool will be refined. The use of this tool will be evaluated in the 
face-to-face and online patient case discussions as a means for consistent and rigorous assessment of 
student performance and as a basis for providing formative and summative feedback. 

2. It may be possible to examine the relationship between student learning preferences and the 
discussion modality (face-to-face vs. online). 
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Introduction 

Handheld computing technologies promise educators the enabling of distributed cognition, with its 
potential of high level computations, immediate access to data, interpretive tools, based on a user 
responsive interface (Multisilta, Henno, Lipiainen and Hamalainen, 200 1 ). In medical education this 
translates to access to clinical information at point of care, scaffolding of the medical decision making 
process, interaction with Web resources and the ability to engineer a dynamic, self directed, learning 
process based on immediate feedback. Examples of applications being utilized in medical education 
environments include, but are not limited to electronic pharmacopoeias (prescription, database type 
programs), e-textbooks, medical calculators (medical formula-clinical computational type programs), 
resident tracking of procedures, patient information and prescription writing. A recent study indicates 
that handheld computers are being used in two thirds of the US Family Practice programs, of which 30% 
require these devices (Cresswell and Parachmn, 2002). 

Several medical schools have developed their own mobile applications, intended to deliver essential 
clinical and basic science information, and monitor learning experiences while students are on clinical 
rotations (Bower and Bertling, 2000; Hammond and Sweeney, 2000; Rosenberg, 2001). The latter has 
been of particular interest as medical practice has increasingly moved to outpatient settings, v^diere 
students are dispersed over large geographical areas in designated clinics (Ruane and Blake, 1990; 
Maulitz, Ohles, Schnuth, Lipsky, and Grealish, 1996; Withy, 2001; Witzek, Koff, McGeagh, Skinner, 
1990). Our experiences in a leading medical school indicate that clinical faculty have specific donands 
for the integration of handheld devices, and have expressed a keenness to use them in idiosyncratic ways. 
Examples include the facilitation of data collection by students, creating portable interactive applications 
for purposes of evaluation and providing context sensitive clinical guidelines during students’ ambulatory 
experiences. 



Objectives 

The purpose of this paper is to reflect on selected aspects of the instructional design process critical for 
the design of mobile applications. In our experience, the success of designing and implementing 
handheld applications in an academic environment is dependent on an iterative process that continuously 
assesses, documents, analyses and responds to the needs of all the constituencies involved, including 
users (medical students), content experts (clinical faculty), critical clinical staff, designers, developers and 
administrators. The context of implementation, the physical and cultural environment of the institution 
must be analyzed in conjunction. We use Smith and Ragan's (1993) model to guide analysis of user 
characteristics. Tessmer and Richey (1997) guides the analysis of the context required for design, 
development and implementation. 

We e xamin e these questions in the context of a variety of mobile learning applications developed in the 
medical school, which requires that students purchase handheld devices: 
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A PDA Patient Log-. This form based patient tracking system is unique for each specialty. Six versions 
have been developed, which allow students to enter essential patient data as they encounter patients. The 
data is uploaded to a central database using AvantGo technology which bridges the Web to handheld 
devices. The data is available for analysis in a variety of ways using a web interface. 

Course information: All the course information, including schedules, faculty contact, course 
assignments, evaluation guidelines and meetings is provided via AvantGo. 

Clinical Guidelines: Disease specific clinical guidelines are delivered by faculty who are interested in 
students’ involvement and learning in particular problems. 

Clinical resources: Resources such as Bioterrorism guidelines, physical exam checklists are available for 
download on a voluntary basis. 

Cases, quizzes, sample exams: Students can view cases, quizzes and sample exams in an interactive 
format on the PDA. 

Evaluation instruments: Several evaluation instruments have been designed should faculty want to 
employ these- single encounter note, one time lecture evaluation, self assessment tools. 

An Iterative Needs Assessment 



While the user and contextual needs of mobile computing founded on the laptop model have been 
addressed in the literature, similar analyses for handheld computing are missing. In the following 
analysis, we present critical questions which must be examined prior to, during and after the design and 
implementation of handheld applications. 



Analysis of Users 



A. Medical 
1 . 
2 . 

3. 

4. 

5. 

6 . 

7. 

8 . 

9. 

10 . 
11 . 



Students 

Prior knowledge: What is the students’ level of expertise in using handhelds? 

What are students’ perceptions of handhelds for their present and future learning? 

What are students’ perceptions on the fiiendliness of handheld devices? 

What kinds of expoiences have students had with technology in the past? 

What are some particular applications that students are already familiar with and find them 
useful? 

What is the students’ lifestyle concerning their clinical rotation? Do they feel connected with 
the faculty and peers during their clinical experiences? 

Is access to clinical information readily available? 

Are faculty or residents available when students have questions? 

What are students inclined to do about technical support? 

What kinds of learning styles do students have? 

What are the specific factors which impede students’ learning? 



B. Clinical Faculty 

1 . What are the perceptions of clinical faculty regarding handheld computing? 

2. How do faculty perceive their knowledge of handhelds and their benefits in medical 
education? 

3. Do faculty think of handhelds as fi'iendly devices which adapts to their needs? 

4. Have faculty embraced technologies in the past? 

5. Have faculty had prior satisfactory experiences with technology? 
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6. What are the specific concerns of handheld computing expressed by faculty? 

7. What are the training and technical needs expressed by faculty? 

8. What are the staff needs expressed by faculty? 

9. Is there a relationship of trust between faculty and administration? 

10. Are faculty open to restructuring the teaching as necessitated by the use of handhelds? 

1 1 . Do faculty perceive handheld technology as efficient and time saving? 

C. Administration 

1 . What are the goals of the administration in implementing the handheld requirement? 

2. What kind of support- financial, infi'astructure, technical, moral, and incentives is the 
administration willing to provide to faculty and students? 

3. Is the administration concerted in its effort to implement the technology? 

4. What is the administration’s overall vision in how the technology will be integrated? 

5. What kind of leadership is available at the administrative level to guide the implementation? 

6. What is the mix of expolise at the administrative levels? 

7. Is the administration willing to restructure education as handhelds become entrenched? 

8. Does the administration have a long term plan and support for integration of handheld 
technologies in education? 

Support staff (clinical) 

1 . What are the perceptions and knowledge of handheld devices among the critical staff for 
faculty? 

2. What kind of support would the staff need to assist faculty and stud^ts in implementation? 

3 . Can handhelds be used to make staff time more efficient? 

4. Do the staff feel capable to respond to changes fomented by the handheld technology? 
Analysis of Context 

Tessmer and Richey (1997) consider context as "complex, multifarious and enveloping" (p. 87), but also 
containing resources that can be exploited. According to the authors, context is a multilevel body of 
factors in which learning and performance are embedded. Indeed this is true of implementing 
technologies which have the potoitial of not only vastly benefiting patient care, reducing the number of 
medical errors, but those v^ch can potentially question traditions of teaching and learning during clinical 
apprenticeships. With this preface we identify the following social, political, physica and cultural 
contextual factors as significant in the implementation of handheld technologies. 

1 . Are users provided physical support to use their handhelds? This includes network stability and 
access, equipment to sync their PDA's and wireless access if necessary. 

2. Are there formal of informal support structures to enable effective use of the handheld technology, to 
accommodate all levels of knowledge and motivation among students? For example, do students 
have access to role models in residents or faculty? Are faculty supportive of PDA's? Does the 
system ensure an instructional model which includes positive reinforcement, feedback to students? 

3. ^\^t kind of technical support is available to students? Has the staff been expanded to provide 
scaffolding during heaviest implementation times? 

4. What is the culture of the use of technology in the institution and its affiliates? Is the technology 
being embedded in a culture of change and successful past implementations? 

5. What is the nature of institutional framework within A\iiich the innovation is entrenched? Is it 
decentralized, or centralized? Is there adequate communication among the various groups and 
entities regarding the implementation? 
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Discussion 



Affordability, combined with the vast number of freely available tools to enhance clinical reasoning, 
make the handheld an attractive technology for medical education. However, as evidenced by prior tales 
of technology advancement, successftil implementation lies in a cognitively rooted examination of the 
potential of technology, an involvement of all the constituencies involved in the teaching and learning 
process and an ongoing assessment which responds to the changes that are inevitably fomented by 
innovation. The needs and context analysis provided above illustrate the complexity of implementing 
handheld devices with the intention of accomplishing measurable behavior changes during clinical 
apprenticeships, while also pointing towards resources which can be exploited to enhance implementation 
effects. 
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Introduction 

The use of cases in medical education is as old as the field itself, with a rich diversity of case methods 
available for adoption. From clinical experiences to virtual reality representations (Grundman, Wigton & 
Nickol, 2000), case-based teaching accomplishes a variety of objectives; students gain familiarity with 
common medical illnesses, are able to construct an adequate differential diagnosis, learn to think like an 
expert, acquire professional attitudes and behaviors and enrich their knowledge structures about the 
pathophysiology of disease. With the advent of on-line instruction, clinical faculty have begun to 
experiment with on-line cases, each with its unique set of instructional strategies. The purpose of this 
paper is to present the design process used to generate interactive web based case presentations in a 
leading medical school. 



The Need for Effective Case-Based Education 

In 1995, Irby conducted a thematic review of literature, based on 101 research articles in undergraduate 
and graduate medical education in ambulatory care settings. Based on the findings, he characterized 
education in ambulatory care settings as symptomatic of variability, unpredictability, immediacy and lack 
of continuity. Students saw a narrow range of problems in a single clinic, and experience limited 
continuity of care. Few cases were discussed and examined with attending physicians. Case discussions 
were short, with little teaching and virtually no feedback. The studies included in the review suggested 
that some students lacked interviewing and physical exam skills and management of psychosocial issues. 
In a separate qualitative study, Irby (1994) idratified five principles of experiential learning in clinical 
settings derived from exemplary strategies used in teaching rounds: anchoring instruction in cases, 
actively involving learners, modeling professional thinking and action, providing direction and feedback 
and creating a collaborative learning environment. Thomas (1992) identified key concons of students 
who had interacted with cases in a family medicine course: students felt most confident about skills 
acquired in relation to diseases with a limited number of k^ symptoms, signs and treatments, and less 
confident about diseases with many symptoms and treatments. The level of complexity in cases was thus 
a k^ issue for clinical faculty. The tutors emphasized that cases must be based on real patients, include 
most of the signs and symptoms of the disease, contain one or two foci, have nodal decision making 
points, emphasize clinical reasoning, reinforce prior knowledge and pamit transfer to other cases. 

Novice to expert research in medical education has yielded a body of findings that can be extensively 
utilized in the design of cases. Gruppen (1997) proposes a framework for ambulatory clinical education 
based on empirically drawn guidelines from cognitive psychology research. The author emphasizes the 
importance of context, students’ need for transferable knowledge, the importance of balancing depth and 
breadth of knowledge, and the role of prior knowledge in problem solving. Research on compute based 
and on-line cases has demonstrated the effectiveness of such cases in promoting poTormance and learning 
efficiency. Lyon, et al. (1991) report on PlanAlyzer — a computer-based, self-paced, cased-based system 
for anemia and chest pain diagnosis. The longitudinal study over three years demonstrated time savings 
resulting from the computer based versus text based cases while accomplishing the same level of mastery. 
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It is clear from a sample of research that in spite of the rigorous training that medical students receive 
during undergraduate years, the nature of the field defies grasp by novices- there are gaps in knowledge 
and intellectual skills which could be addressed with improved instruction. Case based education 
delivered via technology could fill gaps, lapses and enhance the development of clinical expertise among 
medical students. Indeed, such efforts in various medical schools have added to the body of knowledge 
on approaches to case based education (Bergeron, Sato, Bbeid, & Rouse, 1995; Sutyak, Lebeau, Spotnhz, 
O'Donnell, & Mehne, 1996). 

Empirically Grounded Case-Based Development 

Over the last three years, we have developed web based cases in the following specialties: 
Pathophysiology of Disease (Respiratory and Hematopathology), Pediatric Genetics and Emergency 
Medicine. In addition a case authoring tool has been created to allow clinical faculty to create cases with 
diagnostic reasoning in Family Medicine as a model. All cases can be viewed at 
http://www.medsch.ucla.edu/idtu/projects/frstprojects.html. In designing these cases, we briefly present 
criteria that guided the instruction. 

Objectives: What do the cases intend to accomplish? These may consist of diagnostic reasoning skills, 
solving single complex cases, knowledge acquisition via cases and pattern recognition. Objectives might 
pertain to standardizing the clinical experience of dispersed students, provide cases infrequently seen 
during training, and those which need to be finely discriminated. 

Learner Characteristics: Who is the primary audience? Beginning medical students, third and fourth 
year students or residents? How would the program adapt at some basic level to prior knowledge, 
motivation, technology access, and preference for learning modalities? 

Fidelity: What is the level of realism that is desired in the cases? What is the range of imaging from 
virtual immersive reality to still images which would accomplish the objectives? What kind of 
multimedia options — audio, video, synchronized multimedia are feasible? Are standardized patients 
satisfactory, or real patients should be identified? 

Scaffolding: Several studies have pointed to strategies for successful scaffolding (Hmelo & Day, 1999; 
Jonassen, 1996). What kind of support would expand students’ knowledge networks? How can prior 
knowledge be integrated in the scaffolding? How can it be presented in a transparent, well aligned, 
unobtrusive way? Should this be optional, under learner control, or required? 

Communication: Theoretical considerations might dictate the structure of communication. Problem 
based learning which dominates medical education requires and embedded intervention, where students 
are required to post their hypotheses on diagnoses. Other communication strategies may be external to 
the program, and voluntary. The available technology may dictate what shape or form is assumed by the 
communication. 

Interactivity: What is the depth of information processing that is required of learners? Diagnostic 
reasoning can be considered the highest form of problem solving — at a micro level, however, students 
need to make decisions which allow them to encode knowledge required for problem solving. Input 
mechanisms include recall, recognition, hypothesis generation and differential diagnosis. Input devices 
such as speech recognition software would be a consideration how student thought processes are elicited. 

Case Structure: Much debate has occurred over learner versus program control over the last decade. 
Since we have high ability adult learners as the primary audience, we have largely self-paced, learner 
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controlled case designs. The sequencing of information is another consideration — is this simple 
offering a more linear approach, or complex with many options? 

Instructional Strategies: Is the overall presentation a didactic, expository, or more discovery oriented? 
Are the strategies used supplanted or generative? What kind of cueing strategies are available as 
scaffolding assistance? How does the program adapt to various motivational levels? Is feedback rich or 
minimal? What is the modality of the feedback? Does the feedback offer modeling of appropriate 
clinical decision making skills? 

Metacognition: Finally, what kinds of options optimize students’ self monitoring of performance and 
interaction? Are there explicit metacognitive tools which can be placed to make the case more 
meaningful? 



Conclusion 

In spite of pervasive use of cases in medical education, several studies have identified inadequacies in the 
required skills and knowledge of medical students. While some may be addressed within a larger 
fi'amework of curriculum restructuring, the potential of online cases must also be examined in 
conjunction. This paper is an attempt to comprehaid ways of acquiring medical expertise and use this 
knowledge to generate experiCTtial instruction via web based case simulations. Initial evaluations have 
already indicated the positive perceptions of students using this learning modality. Whether the cases 
have also generated learning gains needs further evaluation. 

References 

Bergeron, B.P., Sato, L., Bbeid, J., & Rouse, R. (1995). Connectionist model for multimedia patient 

simulations with diagnostic capabilities. Journal of Educational Multimedia and Hypermedia, 
4(213): 257-270. 

Grundman J, Wigton, R., & Nickol. D. (Oct Supplement 2000). A controlled trial of an interactive, web- 
based virtual reality program for teaching physical diagnosis skills to medical students. Academic 
Medicine, 75(10), pps S47-S49. 

Gruppen, L. R. (1997). Implications of cognitive research for ambulatory care education. Academic 
Medicine, 72(2), 117-20. 

Hmelo, C., & Day, R. (1999). Contextualized questioning to scaffold learning from simulations. 
Computers and Education, 32, 151-64. 

Irby, D. (1995). Teaching and learning in ambulatory care settings: A thematic review of the literature. 
Academic Medicine, 70(1 0), 898-93 1 . 

Irby, D. (1995). Three exemplary models of cased based reasoning. Academic Medicine, 69(12), 898- 
931. 

Jonassen, D. H. (Fall 1996). Scaffolding diagnostic reasoning in case-based-leaming OTvironmaits. 
Journal of Computing in Higher Education, 5(1): 48-68. 






O 



315 



Relan ❖ 327 



Lyon, H. C., Healy, J. C., Bell, J. R., O’Donnell, J. F., Shultz, E. K., Wigton, R. S., Hirai, F., & Beck, J. 
R. (1991). Findings from an evaluation ofPlanAlyzer’s double crossover trials of computer- 
based, self-paced case-based programs in anemia and chest pain diagnosis. Proceedings of the 
Annual Symposium on Computer Applications in Medical Care, 88-93. 

Sutyak J. P., Lebeau R. B., Spotnitz A. J., O'Donnell A. M., Mehne P. R. (1996). Role of case structure 
and prior experience in a case-based surgical clerkship. American Journal of Surgery, 
772(3):286-90. 

Thomas, R. E. (1992). Teaching medicine with cases: Studoit and teacher opinion. Medical Education, 
26(3), 200-7. 



Biographical Sketch 

Anju Relan is an Assistant Professor of Medicine, and the Director of Instructional Design and 
Technology Unit at the School of Medicine, University of California, Los Angeles. She earned a degree 
in Educatibnal Technology from the University of Minnesota in 1991 . She assists the medical school 
faculty in the design and development of instructional web sites related to the 4-year undergraduate 
curriculum. Her research interests include wd> based multimedia in education, metacognitive strategies in 
case based educaticm and the efficacy of handheld devices in the practice of medicine. 



Address: 60-05 1 Center for Health Sciences 

David Gefoi School of Medicine at UCLA 
Los Angeles, CA 90095 
E-mail: arelan@ucla.edu 

Phone: 310.206.0572 

Fax: 310.267.0320 




328 ❖ Designing Web-Based Case Simulations 




Administrative Strategies for the Online Classroom: 

Support Services for Faculty and Students That Produce Success 

Karen Rhoda, Ph.D. 

Division of Distance Learning, Director 
The University of Toledo 

Mark Fink, M. Ed. 

Division of Distance Learning, Assistant Director of Course Development 
The University of Toledo 

Janet Green, B.A. 

Division of Distance Learning, Assistant Director for Marketing and Enrollment 

The University of Toledo 



The success of the distance learning program at the University of Toledo has been built on a foundation 
provided by its administrative strategies, and by its adherence to a structure that produces quality course 
offerings and student services. Distance Learning at the University of Toledo operates as a centralized 
entity to coordinate and facilitate all distance learning courses and degree programs. It provides faculty 
with comprehensive technical support and training allowing them to concentrate on course content as 
opposed to acquiring high tech skills. Student support provides information and technical support services 
rather than faculty needing to respond to students’ technical questions. That is, UT has structured its DL 
environment to support faculty so that they may focus on providing an airiching learning experience. 
Students may focus on their studies and their learning outcomes because their technical concerns are 
addressed. 

Distance Learning began in 1997 with no start up funding and with no courses in place. The enrollment 
number for the 2001-2002 was almost 8000. For the upcoming academic year enrollment already stands 
at 6600 with a projection that enrollment will exceed 9800. The anytime, anywhere environment of UT’s 
online courses allows students who are remote from campus and also those students who cannot easily fit 
work and family responsibilities into a traditional academic schedule to take advantage of a college 
education. Students may enroll in a large variety of courses ranging from philosophy to computer 
science to business management technology. Course offerings are available from seven colleges with 
online degree programs as follows: 

❖ Certificate and Associate degree in Business Management Technology 

❖ Associate degree in Marketing and Sales Technology 

*** Associate degree in Technical Studies 

❖ Computer Science and Engineering Technology partnered with 1 1 community colleges 

❖ B.A. in Lib^l Studies 

♦> Master’s in Liberal Studies 

❖ M.S. in Engineering 

The effective administration of an increasing number of course and degree program offerings to an 
increasingly greater number of students from an increasingly larger geographic area requires a systemized 
process of operation. Verduin and Clark (1991) emphasize the need for a specific model for success in 
distance learning programs and Scollin and Tello (1999) point to the framework for understanding the 
administrative and academic issues in distance learning necessary for success in distance learning. 
Drawing on that paradigm, success in Distance Learning is structured by means of an institutional 
philosophy that integrates comprehensive technical support and training for faculty and student services 
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into an effective entity. In fact, the North Central Association notes this integration is critical to the “best 
practices for electronically offered degree and certificate programs” (p. 52). 

Concentration on this integration promotes good teaching, effective learning and the broadening of 
students’ intellectual horizons. The focus is on strategies that produce quality courses and student 
services rather than on technology alone. This is the reason that Distance Learning is set up to work with 
faculty in the development of online courses. An instructional designer and an assistant are assigned to 
each faculty member so that the faculty member’s course content and research may become adapted to the 
online arena. Professors’ teaching proficiency and knowledge base is brought to the Web based medium 
as they learn to employ technology to deliver their course content and communicate with their students 
electronically. 

Distance Learning encourages UT’s faculty to respond to the adaptation of course content to the online 
arena in several ways. First, DL Faculty Fellowships opportunities are posted during the spring semester 
requesting the submission of proposals for the development of one or more online courses. Successful 
proposals delineate how a course will be adapted to an online teaching endeavor and what population of 
students will be interested in registering for the course offering(s). Chair and dean’s letters of support 
granting approval for the online course endeavors must accompany proposals. A DL Faculty Fellowship 
Committee assesses the submissions determining the awardees and the DL Director notifies the recipients. 
For 2002, seven faculty members received $6000 each for their successful Fellowship proposals. Also, 
Distance Learning supports conference and research endeavors related to distance learning for UT faculty. 
That is, faculty are reimbursed for conference expenses up to $1000 each by Distance Learning because 
of the vital importance of their professional involvement as we move forward institutionally in this arena. 

It is the tenured faculty whose dedication we admire as they bring their years of experience in teaching 
and research to developing and teaching online courses. They are serious about delivering an enriching 
educational experience to their students. We rely on their communication with us to determine the 
positives about their online development and teaching ventures and also the elements that should be 
changed. 

Second, Mark Fink, Assistant Director for Course Development, leads the DL Design Team adhering to 
NCA’s accreditation standard for the provision of “an ongoing program of appropriate technical design, 
and production support for participating faculty members” (p. 56). The strategy is to produce courses in 
which learning is dynamic and interactive and that “are organized around substantive and coherent 
curricula which define expected learning outcomes” (p. 52). In assisting faculty in course development, 
the Design Team creates a complete course for the faculty member including the design elements, web- 
based content pages, photography, graphical imaging, original artwork, animation, video, audio, and other 
related services for the course. 

Third, Janet Green, Assistant Director for Marketing and Enrollment leads DL student services. The 
philosophy herein is that of promoting a feeling of affinity with the University for students wherever they 
may be. This affinity is built “by giving students a single point of contact for everything from advising 
and registration to technical support” (Rhoda as quoted by Lozenzetti, p. 1). Students who develop a 
sense of affiliation are more likely to stay the course. 

Distance Learning is systemized in the processes that develop a course for online delivery and in the 
provision of student services. This figure provides a graphical display of the process by which Distance 
Learning facilitates and coordinates DL courses for UT: 
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Support Elements for Faculty: The DL Design Team 

Within the Design Team, several considerations were made in determining how to structure faculty 
support. As Distance Learning is responsible for providing distance education opportunities for all 
colleges, dividing instructional design work by college appeared logical at first glance. After investigating 
this possibility, it was determined that there could be a transfer of knowledge across disciplines. For 
example, if an Instructional Designer working with a faculty member in the physical sciences designed a 
problem based learning (PBL) instrument, the instrument — or a component of that instrument — might 
prove beneficial in a social science course. But this cross-disciplinaiy approach to design still had to be 
managed. Due to faculty constraints on time, it was necessaiy that the Design Team be organized so that 
faculty know that they could contact one individual in DL to have any issue resolved. While individual 
Instructional Designers may not be responsible nor have the answer, they are responsible for seeing that 
the person responsible for the issue within the division is informed. This approach to the organization of 
Distance Learning has formed the team approach to management, and in fact, we call ourselves the DL 
Team. 

While it appears that the Design Team requires a larger staff, the staffing is comparatively less than many 
institutions with fewer courses. DL has not had the luxury of easily adding to the team. When a faculty 
member is confirmed as teaching a distance learning course, the Director of Distance Learning notifies 
the DL Leadership Team. This initiates faculty support services. The Assistant Director for Course 
Development assigns an Instructional Designer and assistant to each faculty member. The Instructional 
Designer is required to have formal knowledge in learning theories and works in partnership with a 
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Visual/Digital Artist. Next, the Instructional Designer schedules an orientation meeting with the faculty 
member explaining the services that are provided by Distance Learning. Always, the faculty member is 
treated as Ae content expert, the mentor, the instructor, and the director of the course. The orientation 
includes training on the course portal used (WebCT) and “homework.” Homework can include such 
items as asking the faculty member to consider new possibilities for delivery media that will facilitate 
course interaction and presentation, or discover what will not transfer well from the traditional classroom. 
While printed materials are given to all faculty at this orientation, each meeting varies based on the 
characteristics of the faculty member, the course, and the objectives of the instructional materials. 
Meetings with the faculty member continue on a regular basis to provide continuous quality 
improvements. 

Unloading faculty from the burden of acquiring high tech skills has been imperative to the success of 
Distance Learning. While faculty are certainly free to “do it all” if they choose to do so, they have 
overwhelmingly appreciated the DL Team’s services and technical knowledge level. Some of the services 
eliminated entire tasks that faculty do not time to learn or effectuate, while other services provided more 
efficient methods of completing the same tasks. Faculty do not have to load students into each course 
they instruct. Rather, the DL Network Team loads students for the faculty member. Rather than enter 
each question of an assessment instrument into a question databank, faculty can either ftp an entire quiz to 
the portal or send their quiz (in Word, email, fax) to their Instructional Designer for entry. Nor do faculty 
have to know how to build all content within including the portal of their course. Rather, faculty meet 
with their Design Team, explain what they need, and review high tech courses created through a unique 
partnership between the Design Team and faculty member. Too, Distance Learning provides a web-based 
faculty forum with a chat room, bulletin board, a showcase, and a series of content areas providing the 
latest research in teaching via distance. This forum became particularly useful for faculty who teach 
courses from locations from China, Ghana, and Hawaii. 

The Design Team will meet with faculty in the faculty member’s office providing the advantage that 
utilizing “the equipment the learners are actually going to use when doing their real projects” and that in 
the privacy of their own office the faculty member “may make as many mistakes as s/he wants without 
that lack of sophistication being widely toown” (p. 44). Faculty members also have the DL Faculty 
Training Lab available for their use. This is the DL computer facility that provides both PC and 
Macintosh computers, scanners, high-speed Internet coimections, graphics tablets as well as a training 
server for faculty to be able to test course material so that an ongoing class site is not disrupted. 

Support Elements for Students: DL Student Services 

While it may appear that student support has nothing to do with faculty development, the reality is that 
students are imperative to success. A student who is under prepared to engage in a distance learning 
course impacts the quality of instruction the faculty is able to provide other students in the virtual 
classroom. Because Distance Learning provides technical support and prerequisite skills to students, 
faculty spend less time teaching non-content skills, and more time interacting with students. Thus, faculty 
are able to concentrate on delivering positive learning outcomes to their students. 

The phenomenal growth of Distance Learning has occurred as a direct result of the outgrowth of 
comprehensive student support services. The Assistant Director of Marketing and Enrollment coordinates 
student services so that Distance Learning provides immediate responses to students’ questions, problems 
and concerns from the coordination of admission and registration to technical questions and problems. 

An informative letter that includes ID and password directions is mailed to each DL student before the 
start of each new term. Students are provided with: a web-based “help” and support area, a DL email 
address, a toll-free telephone number, a video tape that will be mailed to them at no charge, a tech help 
desk operated evenings and weekends, and software is shipped to their homes. Complete information is 
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posted to the web based “help” site that includes frequently asked questions. Library services are online 
for DL students with provision of journal articles that are emailed to students and books that are shipped 
with the label affixed for their return shipment at no charge. Textbooks may be ordered on the 
University’s bookstore web site and are shipped directly to the student’s home. In addition, the Distance 
Learning is developing a knowledge database for the web, a support chat room, a web cam support area, 
and a complete course for students providing study strategies academic success for any course taken at a 
distance. 

Quality services to faculty in the effective production of online courses and quality services to students 
can be accessed in the results of student retention in online courses. For Spring 2001 the drop-out rate for 
the same on campus courses versus online courses in three colleges was similar; 14.64% for the on 
campus courses and 12.50% for the online courses. Too, course assessments fr«m Spring 2002 report 
student satisfaction with the DL tech help support desk. 

Summary 

The essential ingredient to the ongoing success of any distance teaming program is the recognition in the 
university’s administrative ranks that quality services for faculty and students are a critical factor. This 
recognition brings integrity to the components of assembling a distance learning program that not only 
builds the institution’s enrollment but also produces a productive teaching experience for faculty and a 
quality teaming experience for students. The University of Toledo’s Distance Learning program is 
committed to this principle. 
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The University of Texas System TeleCampus and RLO’s 

In 1998 the University of Texas System TeleCampus began delivering fully online Web-based courses 
using a Courseware Management System (CMS) developed by UOL Publishing (now called VCampus 
Corporation). In 2001, after a lengftiy RFP process, the TeleCampus selected Prometheus for its next 
generation CMS. Less than six. months later, Prometheus was acquired by yet another CMS vendor. 
Blackboard. This experience brought home the reality that the courseware management market is 
characterized by consolidation, acquisition, and rapidly changing technology. 

Higher Education Drivers for RLO’s 

The Web offers a fundamental shift in the way instruction is delivered. When contrasted with face-to- 
face instruction, Web-based instruction is differentiated by asynchronous delivery and a focus on mass- 
customization versus mass communication of content. In the real world of budgets, Web-based 
instruction requires a higher initial investment in dollars and time; these costs are offset by a lower 
delivery cost that is more scalable than a physical setting. In order to protect these large investments; in 
Older to avoid building a course over and over again, with large construction costs at every iteration; in 
order to access courses that can be quickly repurposed for changing audiences, the TeleCampus decided 
in 1997 to build content in a modular fashion. Although the term was not in widespread use at the time, 
the TeleCampus selected Reusable Learning Objects, or RLO’s, as its primary instructional template. 

RLO’s and Standards 

RLO’s are intimately involved with standards. A learning object that cannot be reused in a different 
delivery system (as well as in different content) is worthless. However, wiiile “standards” is a hot topic 
and hyped by vendor marketing, it is important to distinguish between standards, which are universal 
definitions agreed upon by accrediting standards bodies such as IEEE or ANSI or ISO, and specifications, 
which are suggested implemaitations which may develop into accredited standards. AICC, IMS, and 
SCORM are specifications for RLO’s. There are no real or de facto standards for RLO’s at this point in 
the same way that pure technical standards such as XML, J2EE, and RDF already exist. 

Major Specifications 

AICC was the first specification for the reuse of learning content. AICC evolved out of the aviation 
training industry and CD-ROMs where aviation training (which was enormousfy expaisive to develop) 
needed to be repurposed (as Boeing brou^t out the 737 and then the 747 and then the 757) and as 
training delivery mediums evolved (from video-discs to CD-ROMs to Web-based training). 

IMS (Instructional Management Systems) is a global consortium focused on developing and promoting 
open specifications for facilitating online distributed learning activities such as locating and using 
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educational content, tracking learner progress, reporting learner performance, and exchanging student 
records between administrative systems. IMS has published XML bindings and schema for functions like 
question & test interoperability, content packaging, and the systematic categorization and tagging of 
learning objects via metadata. 

SCORM, the Sharable Content Object Reference Model, is the most recent specification, and it attempts 
to unify AICC’s work in architecture, IEEE's Learning Object Model, and IMS’s work in metadata. 
SCORM is a project of the Advanced Distributed Learning Network, part of the Department of Defense, 
and has alfea(fy sponsored several “plug-fests” whereby vendors can demonstrate their claims of “plug 
and play” interoperability. SCORM provides a reference model that defines a Web-based learning 
“content model” and offers a practical bridge from emerging technologies to commercial implementations 

These efforts do not represent standards, but rather they are the groundwork for future standards. The 
LTSC (or PI 484. 12) committee of IEEE will likely adopt SCORM or a SCORM-derivative specification 
as the RLO standard to provide modular, interoperable content that can respond to the various and 
changing needs of institutions and students. 

Impact of RLO’s on a CMS 

A CMS is merely one component of a web-based instructional system that takes content (usually from 
faculty, but increasingly from publishers offering online alternatives to their print products), hosts and 
delivers that content on hardware (from a campus Web server — or an outside Application Service 
Provider) in a system that provides structured access (which can be as simple as the ASP technology 
bundled into Microsoft's Internet Information Server or as complex as a full CMS) and is augmented by 
additional services such as e-commerce, discussion and chat applications, and tutoring services. 

The UT TeleCampus purchases outside services in all of these areas. In three areas — content, hardware, 
and services — local alternatives are available. However, in one key area — content management 
software — an outside vendor provides sole-source functionality. Given the recent experiences in CMS 
vendor change and acquisition, the early decision to use RLO’s appears fortuitous, even though it was not 
necessarily far-sighted. 

Bacl^roimd of RLO’s 

RLO’s offer a method for the reuse of content. Wiley has pointed out that Gerard anticipated the 
development of RLO’s in 1969 when he stated that “curricular units can be made smaller and combined.” 
The current use of the term originated in the corporate training arena toward the end of CD-ROM 
domination as the primary delivery mode and is generally credited to Wayne Hodgins in a 1994 paper 
delivered to the Computer Education Management Association (CedMA, a group of engineers from major 
software companies like Microsoft, Autodesk, and Adobe who were trying to design help systems for 
their software products). As database vendors positioned their products to be appropriate vehicles for the 
delivery of dynamic content over networks, and as object-oriented databases developed the capacity to 
deliver multimedia files natively (rather than as the typical relational database field type of BLOB: 

Binary Large Object), and as software companies saw their product life cycles accelerate, IT training 
organizations such as CEdMA began to espouse the concept of developing specific, modular learning 
“chunks” to teach a specific concept. These modules could be used over and over in software training 
courseware (for example, the File/Save command may remain stable over several lifecycles, eliminating 
the need to develop a training module for this particular command with every software iteration). 
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A Theoretical Model for RLO’s 



After the introduction of the term, cognitive psychologists and instructional designers began to define 
specific characteristics of RLO’s based primarily on concepts from object-oriented programming 
research. 

RLO’s Are Self-Contained 

RLO’s do not draw on outside resources. For this reason, a learning object must not be dependent upon a 
content management system — although it must be able to communicate with all standards-compliant 
management systems. In software terms, learning objects are encapsulated, modular and discrete — all 
communication to an object it is done via messages, and everything an object can do is represented by its 
message interface. 

RLO’s Are Internally Consistent 

They are abstract in that they contain both data and operations on that data. In software terms, learning 
objects are persistent — ^they can be moved around and used in different ways — but they do not change: 
RLO’s are interoperable. 

RLO’s Are Externally Referenceable 

This is the promise of learning objects as exemplified by MERLOT — that a common library of learning 
objects can be developed and shared. The software term for this is “discoverable,” and the use of 
metadata is the implementation. 

RLO’s Provide an Instructional Event 

Gibson adds this requirement to the definition as a feedback mechanism A\4iich provides delineation 
between “data” and “learning” and which serves as a major component of the Cisco RLO model. 

An Applied Model for RLO’s 

Drawing on Clark's work in outcome-based learning, Barritt (formerly with Oracle and now with Cisco) 
has proposed a model where smaller objects are aggregated, bracketed with an overview and a summary, 
and verified by an assessment. The smaller objects may be information objects that generally mirror 
Bloom’s taxonomy: concq)ts, facts, procedures, processes, and principles; or they may be practice 
objects: reinforcement activities that give the learner the opportunity to apply skills or knowledge and 
often provides feedback, remediation, coaching, guidance, and mentoring. Practice objects are not the 
same as but must map to assessment objects; both practice and assessment objects must match cognitive 
level and learning objective. 

The overview provides a single learning objective, an outline, and may include an outcome (performance) 
scenario. The overview typically starts with an introduction that relates the importance of the RLO, 
includes prerequisite knowledge, and is organized by a system-generated outline of information objects 
contained in the RLO. The summary includes a review (that is tied to the scenario, assuming one was 
used) and may include additional resources and a “next steps” statement that provides the transition to the 
assessment. The pro- and post-assessments includes consideration of factors such as pass/fail thresholds 
and partial credit, retries, and the relative weight of assessment items. 
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In this model, based on Merrill’s Component Display Theory, assessment objects must test an individual 
concept. In order to test an integrated, real world performance, the assessment must be tied to the 
summary; however, that architecture is not supported in the model. Wiley, though, has pointed out that if 
scenarios are used in both the overview and summary, and if assessments are ass<x;iated with them via 
these scenarios, learners can pre-test on the integrated performance instead of taking the assessments 
associated with each individual concept. 

A Practical Implementation of RLO’s 

RLO’s are more than theoretical models. The University of Texas System TeleCampus recently tested its 
RLO strategy >^en courses were migrated from VCampus to Prometheus. And the strategy will be tested 
once again when we migrate content from Prometheus to Blackboard. The first step — a correlation of 
specific functions between the two platforms — ^produced a trivial one-to-one mapping. The specific 
movement of course content, originally develops as connected HTML pages in die proprietary VCampus 
database, required the extraction of those pages and assets, repackaging the individual pages into self- 
contained lessons, and then the addition of code to provide internal navigation. The end product was 
RLO’s that can be quickly implemented and reused in any courseware delivery platform. 

What Is (Is Not) an RLO? 

An HTML page — or even a series of pages — is not necessarily a learning object if no event takes place, 
no feedback occurs. However, a single HTML page could be a learning object, in the case of a Java 
applet with a self-contained simulation. This typifies the current debate on RLO granularity: how big (or 
how small) should an RLO be? By using the software concept of inheritance, object classes can de 
defined that share some or all of the characteristics of related classes. In instructional terms, learning 
objects are grouped so that they are independent of but build on previous objects. In the Cisco model, 
granularity is addressed by fixing the number of information objects at 5 +/- 2; rather than being simply 
an arbitrary number, this figure is tied to research into the upper limits of short-term memory (7-9 
“chunks”). In RLO’s, size doesn’t matter as much so long as we use the hierarchical property of objects. 

The Future of RLO’s 

Most of the efficiency interest in RLO’s has focused on their ability to be assembled in an automated 
fashion. However, in Cisco’s model, students cannot self-assemble RLO’s because they don’t know what 
they need to know in order to construct the learning object; that is, students do not have the analysis 
knowledge required to evaluate the RLO’s, nor the synthesis knowledge to assemble the information. 
Instead, information objects must be paired with knowledge objects that identify the instructional function 
played by an information object in order to automate the creation of RLO’s. Faculty will play the pivotal 
role in creating these knowledge objects for useful instructional sequencing. 

Faculty already play the pivotal role in manually assembling RLO’s into meaningful instruction through 
metadata searches. A third approach suggests that subjective metadata, provided by communities of 
practice and working from research in online self-organizing social systems may provide a third and more 
efficient method for assembly of RLO’s; learners who share key chacteristics (learning style, educational 
experience, desired outcome objectives) recommend the “XYZ” RLO, thereby forming communities of 
learning recommendation. 

A critical characteristic of objects is polymorphism: the relevance and meaning of the object is defined 
by the context of its use. The central tenet of software object design is a specialized form called runtime 
polymorphism: the existence of an object is determined at the instant a learner needs it based on the 
learner’s previous actions with other objects. In simplest terms, an RLO will determine that a student 
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needs not just a remedial lesson (based on a failed assessment) but that the learner needs a video instead 
of a textual explanation. This requires the development of equivalent objects that teach the same concept 
but differ in language or media type (for example, audio files versus transcripts). This Holy Grail of 
adaptable RLO’s requires continuous improvement practices in RLO development. 
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Introduction 

Web site usability testing throughout the course development cycle saves time, reduces development 
costs, and ultimately results in a better learning experience for students. During 2001, a team of 
instructional designers, technologists, media specialists, and administrators at the Penn State World 
Campus created a new course design template for online courses and then conducted a web site usability 
study to assess its new features. This paper describes the usability testing process and provides 
recommendations for enhancing the usability of online courses. 

Methodology 

Usability studies can be conducted without great expense or expenditure of time since the majority of 
usability problems can be uncovered by a group of five or fewer participants. In fact, testing with more 
than five participants results in diminishing returns (Nielsen, 2000), and small tests that are conducted 
throughout the development process can reveal significantly more problems than a single test involving a 
large group of users (Krug, 2000). In our usability study, six of the 12 participants were assigned the role 
of prospective student while the other six participants acted as enrolled students. This division of roles 
and assigned tasks was necessary since the course template was originally designed to serve both 
audiences. 

Different usability test methodologies yield information about different aspects of a site, and tests using 
multiple methodologies provide more information than those incorporating only one (Pearrow, 2000). 
Questionnaires provide background information about test participants — representative target users — ^but 
do not uncover specific task-related problems with a site. Similarly, card sorts elicit valuable data on site 
organization and labeling, but results can be as varied as the individuals participating in the test. To find 
specific task-related usability problems, users should be observed as th^ interact with the site. Typically, 
participants are assigned a set of tasks to complete while observers note the time on task, the number of 
mouse clicks involved, and whether the task was successfully completed. Additional qualitative 
information can be gathered by adding a “cognitive walkthrough” component, where participants are 
asked to speak their thoughts while completing the assigned tasks. 

We utilized four methodologies in this study: a background questionnaire, a card sort, a cognitive 
walkthrough, and an exit interview. Participants proceeded individually through the usability tests. The 
background questiormaire was designed to help us gauge the computer skills of the participants. In the 
card sort, participants were asked to define and categorize the menu items that we had chosen for our 
course template. The enrolled group was given our chosen menu categories to work with, while the 
prospective group was asked to create their own categories. The menu items and categories were written 
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on index cards for easy organization, hence the name card sort. In the cognitive walkthrough, prospective 
students were asked to perform four tasks and emolled students were asked to perform 10 separate tasks 
within the course web site. For example, the prospective group was asked to find the date of the first 
exam, ^^^ile the enrolled group was asked to navigate to the first page of Module 1 . Participants 
expressed their thoughts aloud ^^flile their comments and actions were videotaped and noted by the 
observers. The data gathering for each participant was completed with an exit interview, which consisted 
of five questions about the site and its ease-of-use. We also requested general feedback and suggestions 
for improvement at this stage. 



Results and Recommendations 

The background questionnaire indicated that the participants had roughly similar computer backgrounds, 
with most identif^g themselves as being comfortable with computes. We found the exit interviews 
particularly helpful in generating solutions to usability problems uncovered in the cognitive walkthrough. 
The card sort and the cognitive walkthrough generated a large amount of information on the usability of 
our course template and were central to improvement of our site design. Due to page constraints, only the 
most significant and generalizable results from these two methodologies will be described here. 

Card Sort Results 

Result 1: Some menu hern wording was unclear. Participants were confused about the meaning of 
some of the menu hems. For example, our menu included an item titled “directions,” which was a list of 
directions for using course tools such as e-mail and the bulletin board. When asked to describe what they 
thought this item was referring to, answers ranged from “how to do something” to “concrete contents” 
with a number of participants simply saying the thle was too vague to accurately describe. In other cases, 
we were using distance education jargon that was clear to us, but not to those unfamiliar with the field. 
One example was the menu item “IL Student Guide.” Out of all the participants, only one knew that this 
was a student guide of general policies for independent learning students; everyone else simply stated that 
they didn’t know what this item was. Furthermore, no participants correctly described the menu item 
name “Online Diversions.” Even menu items that seemed clear to most students were still confusing to 
some participants. For example, while most knew that “lessons” was a link to the course content, one 
individual thought “lessons” would link to a syllabus or schedule. 

Recommendation 1. Be as clear as possible when referring to course components in menus or other site 
navigation. From this study we discovered that names for menu items must be chosen carefully in order to 
provide clear direction. Catchy phrasing (like “Online Diversions,” in our case) and distance education 
jargon should be avoided. However, we do recognize that in certain cases, creative wording is necessary 
to reinforce the metaphor behind the organization of the site (such as the “classroom” metaphor or the 
“personal organizer” metaphor); in these cases, we suggest that a description of each course component 
be easily accessible from within the site. 

Result 2: There was little consistency between participant-generated cathodes. The enrolled group 
was asked to classify 30 menu items using the categories we provided. “E-mail” and “bulletin board” 
were the only two items that all participants assigned to the same category — ^“Communications.” There 
were nine other items that the majority of participants categorized consistently. Most participants did not 
classify the remaining 19 items according to any one model. When the prospective group of participants 
defined their own categories, there was even less consistency. For instance, there were only 8 items that 
more than 50 percent of the participants placed in similar groups. At best, only two-thirds of the 
participants agreed at any one time. This result led us to question the value of grouping similar items 
under headings in the course menu. 
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Recommendation 2. Although collapsible menu categories and nested menu items conserve space and 
are visually appealing, we recommend that such features be avoided in course Web sites. Based on the 
results of this study, we chose to limit the number of items in the main menu so that the menu could be 
skimmed easily. A shortened menu eliminates the need for students to search through our predetermined 
categories to find a specific course component; it also avoids the use of pop-up menus or other dynamic 
HTML that might not be accessible to students with disabilities. To shorten our course menu, we decided 
to include only those links that would be used repeatedly throughout the course. We then created a space 
for links to course components used less frequently (such as orientation materials, viewing tips, and 
instructor biographies) on the course home page. 

Result 3: Certain menu items could be interpreted differently based on user expectations. Our 
course menu contained an item named “sample content,” which linked to an example lesson from the 
course. In the card sort, some participants wrote similar definitions for this item, such as “example of a 
lesson,” showing that they had a similar conceptualization of the site. On the other hand, some had a 
completely different outlook, instead describing the item as a “sample/example of a report/presentation” 
or “example of works like those student is supposed to create.” While these responses could not be strictly 
attributed to the participants’ roles of prospective versus enrolled students (i.e., not all prospective 
students thought this item would be an example of a lesson, and not all the enrolled students thought it 
was an example of student work), we were struck by the fact the results still fell into two different 
perspectives. Similar findings from other menu item definitions caused us to consider the different ways 
prospective students and enrolled students would use the site. 

Recommendation 3. One site should serve only one function. Single-function sites are easier to design, 
maintain, and most importantly, use. We ultimately chose to create two separate Web sites for each 
course, one for the prospective students and one for the enrolled students. Certain items, such as sample 
materials, a full course syllabus with assignments, instructor information, FAQs, and orientation materials 
were relocated to the new prospective student site. 

Cognitive Walkthrough Results 

Result 4: Participants did not always realize that they had completed a task successfully. In several 
tasks, both prospective and enrolled students had actually navigated to the correct web page but did not 
see the information they needed— and so did not think they had completed the task. This happened, for 
example, w4ien the prospective students were asked to find enrollment information and also Mien 
enrolled students were asked to find help downloading software for the course. These results prompted us 
to find ways to reorganize and streamline auxiliary web pages for quick and easy information retrieval. 

Recommendation 4. We recommend that all pages in a course Web site be clearly labeled so that the 
student knows where they are within the course at all times. This labeling must also be consistent 
throughout the course in terms of appearance and terminology. In addition, course Web pages should be 
constructed so that the most important information is written concisely and prominently positioned near 
the top of the page. Extra information should be removed, or if necessary, relocated to the bottom of the 
page or to linked files. In doing so, you might even find that some additional descriptive material is not 
necessary at all! 

ResmiSt 5: Studemts did mot mmderstamd directions on tine course Web site or preferred to operate 
tbrongb trial and error. No students were able to successfully retrieve assigned readings from an 
electronic reserve reading system at the Penn State library. This task involved multiple steps, where 
students were first asked to determine what readings were required for a particular module, and then to 
access one of these readings fi-om the library. Completion of this task would require participants to 
interface both with our site and the library database. We provided step-by-step directions for using the 
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library site, but most students did not use these instructions, even after they became hopelessly lost. As a 
result, we needed to find a way to provide directions so that students used them. 

Reconunendation 5. Directions must be included up front in order for students to use them. As in 
Recommendation 4, we realized that we needed to remove all extraneous information within and 
surrounding the directions for our course tools. We then placed all directions in a central “Getting Help” 
location. In addition, we realized that adding multiple access paths to the directions would help students 
retrieve the information as needed. 



Conclusions 

While instructional designers and Web developers endeavor to “think like their audience” when creating 
an online course, they are so familiar with the information and terminologies used in the field that 
stepping outside of their roles is not easy. Web site usability testing led our instructional design imit to see 
our courses through student eyes, and the data acquired through this study fed an iterative template design 
process. Although distance education providers rarely have additional staff and resources to dedicate to 
projects outside the normal scope of activities, questionnaires, card sort activities, observations, and 
interviews require nothing more than participants, office space, a computer, and traditional office 
supplies. Even these “quick and dirty” usability tests will improve the learning experience as well as add 
to the collective knowledge of the design staff. 
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One question from faculty members teaching an online course for the first time is, “How do you do online 
exams?” The basis for this question lies in the notion that online assessment must follow the typical 
assessment methods of campus classes. But this is not necessarily true. In an online environment, as the 
role of the course instructor changes radically from being the deliverer of content to one of mentor to the 
students, the frmction of assessment, and its corresponding techniques, must also change. 

Online assessment needs to be viewed as an interactive mentoring opportunity that can be employed in 
online courses. This perspective provides instructors with the opportunity to move beyond the rhetoric 
that assessment should be utilized as a teaching tool and not as an evaluation mechanism. To achieve 
this, assessment must now take on a new role. The assessment techniques employed in an online course 
should be based on desired learning outcomes. Then these can be used by the students to evaluate their 
own progress through the course materials while also providing the instructor with evidence of the 
effectiveness of the course materials or indications of content areas that need further enhancement and/or 
development. 

These online assessment techniques should be used to assist in developing an overall assessment strategy 
for the online course. This assessment strategy aids in forming student assessment profiles, which offer 
the instructor a snapshot of student understanding. In order to provide an effective student assessment 
profile, the foundation for drafting this profile must first be constructed by building learning outcomes 
which are based on critical course content and applicable assessment methods for determining a student’s 
conceptual framework of these learning outcomes. How does an instructor and the student know that 
there is an understanding of the learning outcomes? 

Learning outcomes should be measurable through an applicable assessment of that outcome and provide 
evidence of mastery of the learning outcome through student performance. When writing the learning 
outcomes for the course, consideration should be given to five areas of focus. First, critical course 
content should be determined. What must the student know in order to function in authentic situations? 
Second, discern what the students should know or accomplish based on the critical course content. Third, 
decide what evidence is acceptable as proof of knowledge or accomplishment of the learning outcome. 
Fourth, write these learning outcomes in such a way that they reflect the appropriate level of cognition. 
And lastly, determine what student performance will provide this evidence. The selected student 
p^ormance will furnish the method of assessment for both course and modular critical content. 

Assessment should assist the student in changing a behavior and provide guidance to further develop their 
conceptual framework for a particular area of content. In order to accomplish this goal, assessment 
should be a continuous process and should guide the student to mastery of the learning outcomes. An 
assessment strategy thus becomes the foundation for developing the instructional design of the online 
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course. The overall assessment strategy includes four basic levels of development. Level one would be 
to deliver pre-assessmait to understand an individual student’s conceptual framework of a learning 
outcome, which can provide guidance in the development of appropriate learning activities for continued 
learning growth. These activities are part of the learning strategy of an online course, which is the next 
level in developing an assessment strategy. The learning strategy would include the presentation of the 
critical content through interactive, instructional concepts and activities. Level three would be to 
incorporate short assessment opportunities that should punctuate a course to provide the student with 
performance feedback on these learning concepts and activities. Learning styles should be taken into 
consideration by providing a diverse array of assessment methods to reflect student understanding of the 
learning outcomes. This provides the opportunity to fill in the gaps discovered during the pre-assessment. 
Opportunities for relearning and reassessment should be available to students. Finally, a post-assessment 
provides evaluation of the overall student performance and indicates ultimate mastery of the critical 
content and the ability to auth^tically incorporate the content into appropriate situations. 

Assessment is not only the device through which the student gains feedback on their learning progress 
and identifies achievement through mastery, but it also provides feedback to the instructor regarding 
successful course content delivery methods and techniques. By monitoring the evolvement of the student 
assessment profile, instructors can redesign assessment methods and strategies to meet the needs of 
students and provide an accurate account of a student’s conceptual framework. 

Online course delivery provides an organized and systematic approach to assessment. A variety of 
traditional testing methods are available through the use of digital exam building features. Assessment 
options are available in a medley of traditional formats with the options to pool questions and control the 
delivery of the material. These traditional methods should only be a small component of the overall 
assessment strategy for the online course. Keeping in mind that the learning outcome should be assessed 
using an applicable assessment technique, online delivery also provides an environment, which is 
conducive to incorporating a diverse array of assessment techniques. A wide range of instructional 
technology features and tools are available in the online setting to deliver assessment. This flexibility of 
delivery allows for a more student-centered approach to assessment and feedback. 

As part of Northwest Missouri State University’s effort to assist faculty in the development and 
enhancement of their online courses, the Center for Information Technology in Education has been 
working closely with faculty as they discuss and develop new perspectives as to how assessment may be 
conducted in an online environment. The Center has been working closely with faculty to integrate new 
assessment techniques and develop an overall assessment strategy for their online courses. Additionally 
methods for addressing the issues of academic dishonesty and conduct in an online course are being 
examined as faculty design and develop online assessment strategies. This presentation will provide 
participants with concrete, practical examples of a broad range of useful and diverse assessment 
techniques and strategies that may be employed across a variety of online course subject areas. These 
include reflective summaries, projects, papers, presentations, concept maps, peer evaluations, chat 
interviews, and a variety of other techniques. 
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Purpose of the Session 

This session will review the sources of knowledge and information in distance education, and will present 
a matTK, and several criteria for critically analyzing and evaluating them. Furthermore, the audience will 
receive instruction on how to use specific tools for managing their personal knowledge-base about 
distance education. 



Inibrmation Explosion in Distance Education 

In recent years, there has been an explosion of information about distance education. Many new academic 
journals, trade magazines, websites, and books have been added to the knowledge base of the field. These 
new sources of information have been a boon for academicians, practitioners, and students in the field. A 
few years ago, there was a dearth of knowledge about the field. Today, the sheer number of sources, 
variety of modes of knowledge representation, and media used has created a challenge for those who want 
to keep current about the field, and r^nain up-to-date. 

What’s more the information about the field is now presented in a variety of sources ranging from in- 
flight magazines to exclusive professional and academic journals in specific fields of studies other than 
distance education and educational technology. Articles about effective means of communicating with 
distance learners, or creating more accessible instructional websites are in journals targeted to chemists, 
nurses, engineers, and many others. 

Newspapers throughout the country have discovered distance education and e-Leaming. Since 1997 an 
increasing number of newspapers are publishing stories about new programs offered by colleges and 
universities, faculty involved in distance teaching and students engaged in online learning. 

Technology companies, such as, Cisco Systems, Hewlett Packard, Apple, IBM and Microsoft have 
developed websites with information about teaching and learning at a distance. Furthermore, in the past 
six years the number of companies specializing in distance and electronic learning has dramatically 
increased. These companies have not only survived the extremely unfavorable market conditions for the 
dot. corns in 2001, they are growing and thriving. Their frequent announcements, upgrades to their 
products, mergers and acquisitions, or in a few cases, demise add to the avalanche of information created 
every day. In addition, the increasing number of conferences, online seminars, webcasts, 
videoconferences, and CD-ROM databases make the scene indeed crowded. 
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Drowning in a Sea of Information 



According to senior researchers Peter Lyman and Hal Varian at the University of California, Berkeley: 
“The world’s total yearly production of print, film, optical, and magnetic content would require roughly 
1 .5 billion gigabytes of storage. This is the equivalent of 250 megabytes per person for each man, woman, 
and child on earth.” 

The challenge for the professional and the practitioner is not only keeping track of these sources, but 
assessing the reliability and validity of the knowledge and information put forward. What is the base of 
the information presented? Is the narrative based on research, is it opinion based on years of practice, or 
opinion based on conjecture in the spare of the moment? As we are bombarded with new information, 
how do we know how good, solid, reliable, and useful it is? 

At Distance-Educator.com we constantly receive, review, and assess information for presenting on the 
website and the Daily News. Our experience shows that contrary to common wisdom there is an inverse 
relationship between the amount of information available and the knowledge to be had. In other words, 
information must be analyzed, assessed, and categorized in a specific cont^ to be useful to the end-user. 
At this standpoint, knowledge is useful information. Information that is not rated and is out of context is 
almost of no use, as its validity and reliability is subject to question. 

Quality of Literature in Dbtance Education 

In years of sorting out information, we have come to the conclusion that there are 5 categories of 
information available to all of us. Those are presented in the y axis of the following table. 

Table 1 . Categories and Sources of Available Information About Distance Education. 
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Roughly, the shades of grey in this table indicate that, for example, refereed publications at the top left 
side of the table have a low amount of uniformed opinion, while listserv readers are often challenged by a 
high amount of uninformed opinion. Conversely, there is a high density of theory driven, and data-based 
articles in refereed publications, while such items may appear only as bibliographies, or references on 
listervs, newsletters, newspaper articles or even some professional magazines. This, in part, represents the 
dramatic growth of the field in the past 7 years, and a need for articles reflecting the creative thoughts of . 
their authors about how the field is developing now, or should be developing in the future. 
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Nevertheless, simplistic accounts of projects have inundated various publications too. In recent years, 
there has been a dramatic increase in the number of articles that are not data-based, or theory driven. 
Gradually, the literature of distance education has become primarily descriptive. Although, diere are 
numerous questions about how the new digital environment could be used for instruction and training, 
hard data to respond to these questions are hard to find. 

New terms, such as blended learning, asynchronous distance education, and e-Leaming have emerged 
without conceptual justification, theoretical grounding, or practical validation. Very often, these terms are 
coined by those who are new to the field and are not aware of its history and conceptual development. 
They observe a certain phenomenon, describe it in an article or two, and others take their word as fact, or 
knowledge. 

Criteria for Quality 

The literature has become confusing, misleading, and fhistrating. In presenting information on Distance- 
Educator.com, and choosing source matwials for our print publications, we do our best to comply with 
certain standards. You might find these standards useful to manage your knowledge of the field. In our 
practice, source materials that offer a positive response to the following questions are given priority: 

1 . Importance: Given the framework of the field, to what extent is the information important to our 
readers? 

2. Relevancy to practice: To what extent the source material is relevant and useful to practitioners in 
the field? 

3. Contribution to the field: To what extent the material contributes to the development of the field? 

4. Author’s credibility: What is the professional or academic standing of the author? 

5. Publisher’s credibility: To what extent the publisher or distributor of the information is credible? 

6. Factual information: To what extent the source material provides factual information and is devoid 
of hype and propaganda? 

7. Editorials and opinions: To what extent the opinion presented in a source material, is informed by 
theory, research and practice in the field. 

8. Origin and validity: ^^at are the origins and valid grounds for the opinion presented? 

9. Theoretical fi-amework: Is the source material grounded in, or oriented toward a known theoretical 
fimnework of the field? 

10. Literature review: Does the source material refer to the known literature of the subject matter? 

1 1 . Grounding on data: Is the source material grounded on data collected from the field? 

12. Accuracy and Validity: Are the reported results accurate and valid? 

13. Internal Consistency: To what extent the theoretical framework, literature review, and interpretation 
of the collected data relate to each other? 

14. External consistency: To what extent the material is supported by other sources? 

1 5. Display and representation: Is the display and representation of information accurate? 

Managing Your Personal Knowledge 

We all have heard and read about how corporations and large organizations manage their knowledge. But 
good knowledge management begins with each individual. Fortunately, as knowledge in the field has 
increased so has the number of tools, and techniques for managing such knowledge and information. We 
have offered the 1 5 criterion above for your use when you select a source matCTial to study about the field 
of distance education. We use these in addition to many years of experience to select news, research 
articles, policy papers, resources, and hundreds of other items, which we think you would find interesting 
and useful. 
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In addition, we are making the Distance-Educator.com website increasingly personalized so you can 
interact with the site, and create your own knowledge base. At this point, we will demonstrate the features 
of the website to you, and will respond to any questions or comments. 
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Becoming a Learning Community: 

A Blended Approach for Adult Education 

Susan Santo 
Assistant Professor 
University of South Dakota 

Introduction 

This paper focuses on the blended learning approach used to instruct a cohort of working teachers in 
South Dakota seeking a masters degree in technology. During a distance course on change management a 
virtual learning community developed among the students, in which they discussed issues and offered 
solutions to one another without being prompted to do so by the instructor. This paper describes the 
technologies involved, nature of the coursework, and learner characteristics. It is my hope that distance 
learning programs for adults in other settings can use a blended approach to bring about similar results. 

Literature Review 

Learning communities are an essential part of education, whether in the classroom or by distance. Paloff 
and Pratt (1999) wrote what is possibly the definitive source on online learning communities. The 
characteristics of such a community are excitement and passion for learning, the creation of knowledge, 
and seeing issues from different perspectives. The result is more knowledge being gained than if students 
had learned alone. Meaning is socially constructed by asking questions, sharing resources, and supporting 
other students. The “keys to success” include honesty and trust, responsiveness, relevance, respect, and 
empowerment. The authors suggest looking at the frequency, relevance, and quality of participation to 
determine if a learning community has been achieved. For example, one indicator would be whether most 
comments are directed student to student, rather than student to instructor. 

Murphy and Cifuentes (2001) see development of a community as a key elenrent for online courses. 
Members must want to improve their collective knowledge and be able to share what is learned. 

Instructors can help students to become members of a community by providing structure and support at 
the beginning of the course and gradually allowing more learner control. Using a collaborative workspace 
and activities such as getting to know one another helped their students to succeed online. 

Clark (online) gives three principles derived from research on virtual communities that can be applied to 
courses: I) online learning communities that are grown by the members (i.e.., students), not built by the 
instructor; 2) having a leader who welcomes and reinforces learners; and 3) using personal stories to build 
community. According to Clark, one of the “hardest things to do in any online community is to get people 
to give information. One reason is that people just don’t naturally think their way of doing things has 
value....” Instructor strategies include “letting lurkers lurk,” asking questions to get conversations going, 
leading by example, reading every post, controlling the environment (e.g., by naming the forums and 
including short introductory posts), encouraging stories (which help build a sense of identity), and caring 
about the learners in the community. 

Youngblood et al. (2001) grouped the tasks of online instructors into four categories; 

1 . Setting the scene (e.g., making students welcome, giving expectations) 

2. Monitoring participation (e.g., keeping online discussion on track) 

3. Facilitating critical thinking (e.g., asking questions) 

4. Promoting student collaboration (e.g., encouraging students to build on each other’s comments). 
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Bauman (1997) offered these guidelines for creating a learning community in an online course: 



❖ Communicate frequently with the class 
Make as much interaction public as possible 

❖ Create a space for informal intCTaction 

❖ Understand the limitations/strengths of technology for interaction 

❖ Ask questions often and interact with students 

Blended learning, which combines traditional and online instruction, is becoming increasingly popular. 
Zenge and Uehlein (2001) see blended learning as including integrated instructional design (with each 
delivery method used to do what it does best), variety, and a consistent framework, values, and concepts. 
Although the authors believe that this approach can lead to huge gains, they caution that “a full 
understanding of the benefits of combining both approaches is likely only over a long period of time with 
further research.” Traditional instruction may be better at social situations, casual interaction, getting 
feedback from othCTS, and discussing emotions. Online learning may be better for information and 
cognitive skills, as well as being more convenient and less expensive. 

Hocutt (2001) identified four criteria for “strategically blended learning”: 1) the components should be 
developed with an awareness of each other; 2) th^ must be consistent in language and style; 3) there 
should be some redundancy; and 4) there must be a smooth transition from one component to another. 
Haythomthwaite et al. (2000) have found that students who fail to make use of coimections such as e-mail 
and chat feel isolated and experience more stress than students who participate in virtual communication. 
Beginning a course on-campus can help in the bonding experience so that students continue to maintain 
these connections online. 



This section describes the blended learning approach used by the University of South Dakota (USD) 
during spring 2002 to instruct a cohort of working K-12 teachers throughout the state who were seeking a 
masters degree in technology and training. The delivery systems used were videoconferencing and 
WebCT. During the videoconferencing sessions, students broadcasted from multiple sites through the 
Digital Distance Network (DDN), a statewide telecommunications network. WebCT is used for all 
distance courses at USD. 

During the preceding fall, students were given an opportunity to travel to Mitchell (at the center of the 
state) to meet each other and the instructors in person during a special introductory session and dinner. 
This was part of our strategy to build community. However, when I began teaching a course on 
organization development, I found that these students required considerable encouragement to participate 
actively at a distance. In the spring these same students took my course on Personal/Organizational 
Transition and Change Management. This time their interaction and intent transformed from simply 
meeting the degree requirements to the enthusiasm and excitement characteristic of a true learning 
community. They began to comment on each other’s ideas and suggest solutions to one another without 
my prompting them. What had changed? 

I followed several strategies for this new course in an attempt to build online community. 

First, I put up the WebCT site for this course over the winter break and gave students access. I posted 
handouts for each topic, a syllabus, and list of readings. Several students began to post online and to share 
their experiences and thoughts regarding the topic and the textbooks before the course had even officially 
started. This set the stage for what was to follow. 
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Approach Used in My Course 
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At each DDN session, vs^ich simulated face-to-face interaction without students having to be at the same 
location, I began by greeting each site and finding out who was there. I always allowed time for a little 
informal talk so that we could get comfortable before the real work began. I gave students the opportumty 
to discuss any problems they were having. Time was also provided before class for students who wanted 
to talk with me privately. 

I asked students to keep all e-mail correspondence to me or other students within WebCT. This eliminated 
some confusion I had had last semester A^en e-mails were arriving fi-om different accounts. 

Students were asked to keep a personal journal on their Palm Pilots (provided by USD) on change issues 
in their lives. They used the fi’ee software Freewrite. Excerpts of these journals were then loaded onto 
their PC and e-mailed to me. I took care to comment on each students’ journal and on their papers. 
Although students did not see each other’s journals, I found that much of what had been written fueled the 
discussions on WebCT. We began by discussing issues related to personal change. Students were stunned 
to find that their reactions toward change were normal and th^ were not alone in facing many difficult 
personal changes throughout their lives. Once they understood the basic concepts about change (covered 
by the textbooks and the handouts), they were able to apply them. Students grew increasingly excited as 
they realized the real-life application that could apply to them right now. 

During the DDN sessions I relied much less heavily on showing PowerPoint presentations and more on 
class discussion. One issue I faced was having seven sites active during the DDN videoconferencing 
sessions. The technology was voice-activated, meaning that I could only see the site w^ere a student was 
speaking (or had most recently spoken). This made it difficult to monitor students’ level of 
comprehension. I noticed that certain sites were more likely to dominate the conversation, thus 
discouraging quieter, more introverted students at other sites. I began to call on less active sites and to ask 
them for comments and questions. As soon as they spoke, the camera would switch to them. I also 
encouraged students during these sessions to address comments and questions to each other and not 
necessarily to me. Sometimes my keeping silent was all that was necessary to get students to do this. 
Rather than calling on each site, I let students respond more informally. 

For group activities, in cases where there were enough students at a site, they worked in the activity 
together and muted their site so that others could not hear. In cases where there were only one or two 
students at a site, I had these students continuing to use the DDN link to work together. I turned the 
volume down and muted my own site so that they had the privacy to work. I then brought everyone back 
together by turning off my mute and waving my arms at the screen! 

In encouraging online discussions about change, I realized that trust was a major issue. Students could 
only communicate fiwly if they trusted me as well as each other. I helped to develop that trust by 
revealing certain things about my own life and how I had reacted to major change. I answered students’ 
questions about myself. For example, I had moved to South Dakota from Virginia and was facing a 
cultural as well as geographical change. Modeling what I wanted from them seemed to encourage 
students to share with each other and to come up with useful suggestions for helping each other to get 
through a difficult change. Because WebCT provided a relatively private area (i.e., only students that I 
had logged into the course had access), studoits felt that our discussions were safe. 

By midway through the course, perhaps the biggest problem I faced was that students were posting too 
much! There were days when I had a tough time keeping up with them. Instructors need to keep in mind 
the enormous amount of time involved in monitoring and encouraging such communities. 

During the last DDN session, studoits presaited on the plans they had developed for change management 
at their school districts. (I deliberately placed the most dominant site last on the agenda, ensuring that the 
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quieter students had a chance to speak). Students were also asked to post handouts on WebCT giving their 
top strategies for a successful change. This enabled students to gain the benefit of each otho-’s work twice 
and comment on it. In addition, students were asked whether their views on change had altered during the 
semester, which led to a fascinating discussion (they had). 

The amount of change going on in my adult students’ lives undoubtedly influenced their enthusiasm 
about this course. School districts in rural South Dakota are having to face a dwindling population and 
make tough choices. Cutbacks, downsizing, and combining have meant that no one’s job is safe. In 
addition, many of my students were undergoing major personal changes that semester. The fact that we 
had now known each other for a longer period of time - two semesters instead of one - helped also. 
Learning communities take time to build. 

Another group of cohort students taking this course during summer 2002 are attending one session at the 
USD campus in Vermillion. We will be using an activity to develop a change joum^ map of a major 
change taking place in their lives or their organizations. It is my hope that this in-person activity will 
encourage the bonds already developing among the students. I am also using group projects by having 
students use the Student Presentations feature on WebCT. This allows students to work in small groups 
through private discussions, e-mails, chats, and by creating and downloading a shared html document into 
WebCT. Once students have finished their projects, I plan to put a hyperlink to each project so that 
students can view and comment on each others’ projects. 

In conclusion, here is a list of strategies that may be helpful to you in encouraging online communities in 
your distance courses: 

❖ Let students into the online site before the course begins. Have a face-to-face session or two, if 
you can, at some mid-point location, and build in some informal networking time. 

*!* Be active in online discussions. This doesn’t mean you should dominate the discussions, but 
students need to know that you’re there and are paying attention. Don’t just post some questions 
and then disappear. Comment on students’ posts and praise outstanding posts. 

*!* Require students to make a certain number of online responses to other students. I asked 
students to respond to at least two other students for each discussion topic on WebCT. Some 
students made many more comments than that. (One particularly empathetic individual wrote a 
response to every single student’s comment!) 

❖ If possible, limit the number of videoconferencing sites. Two to four sites would be ideal. With 
seven, it gets difficult. Make sure there are at least two students per site (preferably more). 

❖ Connect what students are learning to real life. Students become more involved when they are 
able to see the benefit immediately. 

❖ Have students post some assignments so that others can view them and make comments. This 
can be either required or optional. (I used both approaches, dq>ending on the assignment. 
Students could post their work as an attachment in the Discussion board on WebCT.) 

❖ Have a contingency plan. There were times when the technology didn’t work and students were 
unable to connect to the videoconferencing sessions. We dealt with this by mailing them a 
videotape of what th^ missed. Another possibility would be to connect them to a chat room. 



358 ❖ Becoming a Learning Community 



♦t* At the aid of the course, encourage students to think about what they have learned and how 
they related to each other. Ask how they will use this in the future in their lives. Encourage 
them to keep in contact with one another. (I left the Discussion board up for several weeks 
afterwards.) 
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Co-operative Design Fostering Diversity and Interactivity: 
International Oi^anizational Behavior for a Distributed Community 



Debra Sarlin 

Instructional Designer/Researcher 
Rensselaer Polytechnic Institute/SUNY Albany 



What happens when three institutions attempt to improve understanding and foster deeper insight into 
cultural differences? In an attempt to accelerate Mexico and U.S. team building through educational 
projects, interactive problan solving and online learning exercises designed from a multicultural case 
study perspective: Rensselaer Polytechnic Institute in New York State, Tec de Monterrey in Mexico and 
General Motors worldwide Technical Education Program, jointly redesigned and presently cooperatively 
teach a newly created course in International Organizational Behavior. 

When multiple institutions work to widen a global perspective through international acadanic offerings, 
we begin to address key questions related to diversity and interactivity. How, for example, do our ideas 
about difference affect our actions? How do those ideas and actions affect our communications? If ideas, 
actions, commimications, and binary oppositions or the simple dichotomies that shape our language, all 
work to contour our reality — ^what can we do to change our own perspective and promote change within 
our communities of practice? 

Gaviel Salomon (1998) wrote: 

Typically school settings do not offer nearly the opportunities they might for the development of 
self-regulated, high-road learning. If anything, the problem is even more acute for the learning of 
large collective entities like corporations, which characteristically have hardly any 
“metacognitive” awareness of Aeir own learning. The collective that worries about mediating its 
own learning and the learning of individuals within it is far more likely to learn better and foster 
individual learning, (p. 19 of 25) 

Salomon distinguished between programs designed to mediate individual relations and educational 
program designed to mediate change between collectives (Salomon, 2001). In the spring of 2002, 
nineteen students (W=19) enrolled in our pilot course, an intervention determined in its attempt to 
untangle narratives of both personal and group conflict. This course, offered or distributed through what 
we are calling online web-based conferencing was designed to support group and individual 
metacognition. 

Students were grouped by (a) work location in Mexico and the United States, (b) gender, and (c) known 
characteristics, and were then randomly assigned to one of four teams for the semester. Over the term, 
teams communicated and collaborated— building a course-related, problem-based project — using a 
private asynchronous electronic bulletin board, a synchronous text chat ^d email. 

Several measures were used to analyze course communications and group differences. One measure was 
created to determine student attitudes towards Diversity. Success, in this case, is measured as final 
semester grade — scored for each group by the RPI and TEC professors based on projects, participation 
and peer reviews. Activity is an average of each participant’s “hits,” readings, and postings in the online 
environment. Interactivity as engagement is measured here as message or communication complexity in 
the pattern of posing to, then responding to and sustaining with others in a group (Sarlin, 2002) — in this 
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case — ^within a computer-mediated environment. Reliability for the interactivity measure is strong at 
Cronbach alpha = .87. Reliability for the diversity scale is high at Cronbach alpha = .96 

Table number one shows differences in diversity scores for each of the four student work groups as 
significantly different from one another. Differences in the activity level of each group are also apparent. 
Differences in grades is not significant and additionally grades are not normally distributed. 

Table 1. 



ANOVA 





Sum of 
Squares 


df 


Mean Square 


F 


Sig. 


SCORE Between Groups 


288.392 


3 


96.131 


5.150 


.015 


Within Groups 


242.667 


13 


18.667 






Total 


531.059 


16 








ACTIVITY Between Groups 


77975.169 


3 


25991 .723 


2.603 


.096 


Within Groups 


129830.4 


13 


9986.954 






Total 


207805.6 


16 








GRADE Between Groups 


1.451 


3 


.484 


1.347 


.302 


Within Groups 


4.667 


13 


.359 






Total 


6.118 


16 









Figure one reveals differences in each group’s feelings about diversity. Work ahead will determine how 
differences in activity^ interactivity and success “look” in relationship to these values. 

Figure 1 
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Figure two shows the relationship between individual diversity scores, success and activity as 
standardized values. There appears to be a relationship between individual feelings about human diversity 
and individual course success with some interesting outliers. 



Figure 2 



Correlations for Diversity 



|"0*diversity ^^success * 



^activity | 




This pilot work moves towards determining the relationship between interactivity and conceptions of 
diversity in computer-mediated learning spaces. Building learning environments designed to foster 
metacognition and support social relationships online — as well as off in the widening world — seems a 
supportable vision for the work ahead. 
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Overview of the Partnership 

Rensselaer Polytechnic Institute is the oldest technological university in America. RPI’s award-winning 
Professional and Distance Education Program has existed since 1987. This distributed educational 
program, focused on working professionals, has been built on a valued relationships with many leading 
corporations including GM’s MOT program (which includes five sites in Mexico), IBM, Ford, and UTC. 

Tec de Monterrey is the leading technological university in Latin America. With more than 90,000 
students at 30 campuses, their Virtual University transmits courses and programs throughout Mexico and 
Latin America using Satellite, Videoconferencing, Internet to over 1 ,000 sites in Mexico and Latin 
America. Annually Tec’s enrollments are Over 50,000. 

General Motors Technical Education Program (TEP) was founded in 1984 as resource for their 
Engineering division. Currently TEP is part of “General Motors University” a corporate-wide educational 
service group. Initially its focus was North America, that focus is now global. The TEP Mission is: “7b 
provide General Motors with a competitive advantage by offering to the technical community education 
in current and emerging technologies core to our business in partnership with leading universities 
world-wide. ” 

The Rensselaer Plan focuses on global reach or impact — enhancing, extending and strengthening 
partnership activity with other institutions or agencies. Professional and Distance Education at RPI had 
successfully worked with General Motors providing educational experiences for working engineers, 
designers and managers in North America and Mexico. As Tec de Monterrey and Rensselaer share, 
Icommon values — ^the possibilities for a developing a GM, TEC, RPI collaborative working alliance 
seemed evident and exciting. 

Our redesign of an existing, successful course in Organizational Behavior, holds essential content 
constant yet clearly focuses on the important dimensions of cross-cultural communication, 
internationalization and the impact of technology on organizations. To include the perspectives of the 
three partner groups, we worked to develop an instructional design that would function internationally — 
across time zones and culture, which would be “comfortable” for our corporate partner. 

The course design includes interactive elements, cross-cultural team building exercises and a structure for 
collaborative group projects. Our hopes are to expand this partnership in a variety of ways: an RPI/TEC 
joint degree program for Latin America; faculty exchanges; offering or running “parallel” courses; 
locating funding firom US/Mexico government sources to pursue joint research and course development 
activity. Additionally, we look forward to seeking and forming new partnerships based on this emerging 
model. 
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Distance Delivery: Courses, Programs or Both 



Steve Schlough 
Department Chair 
University of Wisconsin-Stout 



Abstract 

This paper discusses the methods and strategies in implementing a viable distance-learning program. The 
main focus will be on the advantages and disadvantages of delivering stand-alone courses versus degree 
programs, based on the experiences of the Communications, Education, and Training Department at the 
University of Wisconsin-Stout. 

Introduction 

There is much potential for growth in distance learning, but the road to success may not be as easy as 
some proclaim. The Communications, Education, and Training Department has a long history in distance 
learning, beginning with offering courses for certification in the Wisconsin Technical College System and 
teacher enhancement. These were originally offered through paper based correspondence and off-campus 
face-to-face offerings. In 1986, the Department began moving into electronic systems, starting with 
Computer-Based-Leaming and Interactive Video. Since that time, the Department has added 
teleconferencing, videoconferencing, on-line, and web-based strategies to its delivery methods. The vast 
majority of the distance courses offered by the Department have to be self-funded. This requires that each 
course have enough students to pay the instructor, plus any additional overhead. As many have learned, 
when trying to attract adult learners the market is there, but so is the competition. The competition is not 
just from other institutions, but the competition created for the need for time for work, family, recreation, 
and civic responsibility (Peterman, 2000). 



Delivery 

The Department has used a variety of delivery strategies. A short discussion of each follows. 

Self-paced paper based correspondence courses were the Departmait’s first venture into distance 
learning. They provided flexibility and did not require the learner to have any special technology. They 
have been phased out and replaced with on-line courses. Surface mail is not used very much anymore. 
The exceptions are mailing textbooks and other materials that caiuiot be distributed electronically. 

Teaching face-to-face courses on Friday evening and all day Saturday at off-campus sites is a long time 
mainstay and still used extensively. The advantages are that it still has the personal connection, travel is 
eliminated for students, and it does not interfere with most learners work schedules. The disadvantages 
are travel time and expenses for the instructor and it is time and location dependait. 

Video tapes are distributed as part of some courses. One particular example is a photography course 
where a set of tapes was developed to supplement the on-line portion of the course. 

Interactive television is used to teach some courses and as a supplement to others. It provides a better 
level of interaction than on-line, it is easier to demonstrate some items and it can reach multiple sites. It is 
still time and place dependent and is not the same as being there. 
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On-line courses offer great flexibility and yet have interactivity. They are time and place independent, 
although the usct needs to have internet access. The technology has some limitations and requires greater 
bandwidth than many learners have access to. 

The future of hybrid systems has great potential. This is the mixing and matching of appropriate 
technologies. Just as the use of one method of instruction in the classroom is not the most effective, the 
same is true of the use of one technology. 



The only stand-alone courses that the Department offers are those that are part of degree sequences or 
have been requested by someone to meet the needs of a specific audience. Occasionally, the Department 
may offer a non-credit workshop and attach a credit option to it. 

The offering of professional certificates is a step between offering a stand-alone course and a complete 
degree program. As the department has developed certificate programs, they have been designed to make 
sure the courses fit within a degree program. 

Degrees in Vocational, Technical, and Adult Education B.S, Vocational, Technical, and Adult Education 
M.S, and Training and Human Resource Development M.S. are offered through off-campus cohort 
groups using a combination of delivery strategies. A Graphic Communications Management B.S. degree 
completion program is offered at one technical college site. In addition, the Department offers courses as 
part of a PhD consortium with Indiana State University. 

Strategies 

The offering of an individual course or professional certificate is sometimes used in areas where there 
may not be enough demand to start a cohort group or test the water to see what the demand is. Due to 
meeting multiple needs, some of the courses in a sequence will have more of a demand than others and 
these may be offered first. As some courses are used to meet the requirements of more than one degree 
program, several of these have been converted to an on-line format. The courses typically follow an eight- 
week format, allowing the courses to be offered Fall, Spring and Summer. Students thus have an 
opportunity to build some flexibility into their program. 

The goal of the Department is to offer programs on campus, as well as offering cohort groups at two other 
locations with staggered start dates. Students in a specific cohort have preference in enrolling in courses. 
After those students are enrolled, other students may enroll if there are additional seats. 

Conclusions 

Based on existing research and the prior experience of the Communications, Education, and Training 
Department at the University of Wisconsin-Stout, the author concludes that; 

❖ Using a cohort group for delivering a degree program is the most effective way to have 
successful distance education programs. Delivering individual courses and certificate programs 
are successful if they can be used in a degree program. 

*•* Although in-depth needs assessments are required in developing new programs, a more 
informal approach can be taken to convert an existing program to a distance program. It is 
important to know who else is offering similar programs. 

❖ Forming partnerships with technical colleges, military bases, professional organizations, and 
businesses has been the most effective marketing tool. These partners need to be involved up- 
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front because they are the ones who can give the best insight to the viability of a program in a 
particular geographic area. In addition, direct mail and limited newspaper advertisements are 
used wlien starting a new cohort. 

♦J» Since the Department has focused its course delivery within the State of Wisconsin, it has 

allowed for the use of an array of delivery technology. The use of face-to-face instruction is still 
prevalent, as all the cohort groups are located within the State. In addition to face-to-face 
weekend classes, classes are conducted on-line, via teleconferencing, and over interactive TV. 
The use of hybrid courses has been used over the past few years. All courses taught at a distance 
have some type of web enhancement and students are encouraged to keep in contact with their 
professors and classmates, via e-mail, phone, and fax. The course. Learning Technology, is now 
being taught one weekend face-to-face, a four-week session on-line, and wrapping up with an 
interactive television session. The key is to fit the technology to the learners needs, not the 
learner to the technology. 

❖ The Department has moved away from a heavy reliance on adjunct faculty to teach off-campus 
courses for degree programs. Although adjunct faculty typically have excellent teaching skills 
and great content expertise, it is important for students to have contact with professors that have 
knowledge of the programs and working with campus. 

❖ Support to the students is the key element to success. In addition to relying heavily on the 
University’s continuing education unit, the Department has a person who, as part of their duties, 
works as a liaison for the students. This person has a master’s degree in one of the programs 
offered at a distance and has taught both on-line and face-to-face courses to off- campus 
students. This allows the individual to provide support and answer a wider range of questions 
than provided by many support people. 

These conclusions may not be transferable to all situations, as the offerings by The Communications, 
Education, and Training Department are presently limited to several degree programs that serve specific 
niches within a restricted geographic area, but the author believes that they will serve as a meaningful 
foundation as the Department expands into new markets. 

“By utilizing the best of both traditional and Internet educational delivery methods, universities can 
provide an amazingly versatile system stabilized by tradition and anchored by quality” (Bothel, 1999, 
p.6). 
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Introduction 

The use of online learning and computer-based training has been steadily increasing in recent years, and 
so too, has the quality and level of adult learner satisfaction with these mediums. Using self-assessments 
in an online learning environment is a strategy that allows learners to test new knowledge, receive 
meaningful feedback, focus learning, and assist in transferring knowledge for use outside of the online 
environment. Instructional designers (or designers) and instructors need to be aware that self-assessments 
are an important tool that can be used to both assess and aid learning. 

The Value of Self-Assessments 

Assessments, commonly called tests or exams, have been used in traditional classrooms for a variety of 
reasons. Self-assessments, when designed properly, can be extremely valuable for learners in online 
learning environments as well. 

Online self-assessments can be designed to: 

❖ Measure the degree to which content is meeting learning objectives 

❖ Provide an opportunity for immediate application of content 

❖ Test learners on new loiowledge 

❖ Provide meaningful feedback to learners 

❖ Track learner progress 

❖ Focus learning 

❖ Make learning fun, interactive, or flexible 

❖ Assist in limiting or providing access to new information 

For all of these reasons, online self-assessments can be part of an effective learning strategy. However, 
it’s important to be aware that the mere mention of the word ‘assessment’, ‘exam’, ‘test’, or even the 
appearance of an online self-assessment can generate ‘learning nightmares’ for some because of previous 
testing experiaices. Negative views of tests often stem from test-taking anxiety, fear of how the results 
will be used, or from unpleasant experiraices with poorly designed assessments. On the other hand, some 
learners may approach an assessment as a challenge, and they may try to ‘cheat’ their way through the 
assessment. In other words, get every question correct without ever reading or learning anything. For all 
of these reasons, it is invaluable to use well-designed self-assessments. 

Types of Self-Assessments 

There are a variety of effective self-assessments currently being used in online learning environments. 
Some of those online self-assessments include exercises and activities such as multiple-choice and 
matching, pre-tests, post-tests, decision makers, and simulations. 
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Exercises and Activities 



Multiple-choice, true and false, fill in the blank, and exo-cises such as matching, or drag and drop, are all 
examples of effective online self-assessments. Though simple in design, these assessments are used to test 
knowledge, add interactivity, and provide immediate feedback to learners. 

Pretests and Posttests 

These tests are often composed of the exercises and activities listed previously. Pre and post-tests are 
valuable when used to measure learning gain, focus learning, provide a customized path, or limit and 
provide access to course materials and resources. 

Decision Makers or Decision Trees 

Similar to “choose your own adventure” or “if, then” statements, decision makers provide an opportunity 
to apply concepts or strategies in a situation such as a typical business setting. Decision makers allow 
learners to evaluate their knowledge of specific content based on how effective their decisions are. For 
example, if the content is based on teaching mangers to retain valuable employees, a decision maker 
activity may present an employee retention problem to the learner. The learner then makes a series of 
decisions attempting to address the employee problem. The subsequent decisions may be different based 
on the outcomes of the initial decisions the learner makes. By the end of the assessment the learner may 
have successfully retained the employee, lost the employee, or retained the employee through a less than 
optimal approach. These types of self-assessments can be extremely effective when meaningful feedback 
is presented after every decision the learner makes. 

Simulations 

The goal of an online simulation is to provide a simulated environment that allows learners to make 
decisions based on situations th^ would face in real-world environments. The consequences for making a 
poor choice or wrong decision in a simulation should be the same as in the real world environment. 
Therefore, if learners can perform well in a simulated environment, then in theory th^ should be able to 
perform the same behaviors in a real situation. One common simulation is that of a flight simulator. 
Learners need to be able to successfully take-off, fly, and land a plane in a simulated environment before 
attempting to do these same activities in a real airplane. Well-designed simulations can provide safe 
environments to learn challenging skills, yet it’s important to remember that simple multiple-choice 
questions may in some instances provide similar learning without the time and cost commitment of 
building a simulation. However, if designed to be re-used, or for a large number of learners, simulations 
can be worth the cost and effort. 

These online self-assessments differ from traditional tests in several ways. Online environments provide 
the convenience of anytime learning, and designers have the opportunity to provide more customized 
learning experiences. For example, a pre-test can provide immediate feedback as to where a learner needs 
to focus, and it can direct a learner to specific content or resources in areas where more knowledge is 
needed. If a learner already has an acceptable level of mastery with other concepts, there may be no 
reason for the learner to complete all materials. Online learning environments also allow designers to 
create materials that are more stimulating and interesting to the individual than less interactive learning 
environments. A learner working in the manufacturing industry could be provided the option of selecting 
activities, examples, and assessments that use settings, problems, and scenarios appropriate to 
manufacturing, as opposed to less relevant scenarios. In addition, online learning environments allow 
designers to provide tips and hints at times when a learner may be struggling with questions or decisions 
that deal with a specific content domain. In addition, with a large repository of questions to choose from. 



370 ❖ Effectively Using Self-Assessments \ 



designers can test learners at appropriate levels, as opposed to taking a “one-size fits all” self-assessment 
approach. 



Best Practices and Tips for Designing and Implementing Self-Assessments 

There are several components to consider when designing and implementing self-assessments. Three of 
those components are feedback, instructions, and placement. 

Feedback 

Feedback is one of the most critical parts of a self-assessment because this is often when learning takes 
place or is confirmed. Learners can complete numerous activities, but it’s the feedback that informs 
learners if they are correct, incorrect, headed in the wrong, or right direction. Feedback can also provide 
hints and tips, point to resources needed, reinforce the learning of concepts, and provide information on 
next steps. Before providing feedback, designers should consider wiien learners will need feedback, how 
much feedback to provide, and what kind of feedback is needed. For example, consider if all, some, or 
none of an activity needs to be completed before feedback is given. There can also be different levels of 
feedback, a feedback strategy often used in simulations. Feedback levels are used when learners attempt 
part of an activity once and get overall feedback that they are not quite correct and should keep working. 
As the learner spends more time in a simulation, and perhaps continues to make the same mistakes, 
feedback would need to get more and more specific until the learner is told clearly what the problem is 
and how to correct it in order to move on. 

In simulations, learners may not need to be told the correct strategy on a first attanpt; however, in other 
activities this may not be the case. In simpler activities such as multiple-choice, learners benefit more 
from detailed feedback that tells them if they are right or wrong and the rationale behind the answer at 
every att^pt. In this case, feedback should be clear and concise the first time. 

In terms of vsiiere to place feedback, designers should determine wiiere the learner will struggle or need 
assistance, vsiiere the critical content chunks are, and then provide feedback in those places. 

In terms of when to give feedback, consider if feedback should immediately follow an assessment or be 
provided throughout an assessment. Nothing is more frustrating than completing a long activity only to 
find out at the end everything was done incorrectly. However, some assessments may be short enough to 
have all feedback come at the end. In these instances, designers can create meaningful feedback 
statements without fear of giving away answers to other questions. 



Appropriate use of color can be a successful strategy in providing feedback. Learners can associate 
certain colors with specific assessment outcomes. For example, many students associate the color red with 
“stop” or “incorrect” and associate the color green with “go” or “correct.” Designers may choose 
toprovide feedback in colored text in order to visually conv^ correct and incorrect feedback along with 
detailed text. It is important to note that some learners may teve difficulty seeing colored text, are 
colorblind, or may find the color red offensive because it sends a strong message the learner is wrong. 

Feedback may also contain graphics to signify correct and incorrect answers. For example, in a simulation 
that deals wiA increasing revenue, there may be a constant chart displaying dollars. When the learner 
makes correct decisions, the dollars increase on the chart, when the learner makes poor decisions, the 
dollars decrease. Unfortunately, it can be difficult to find symbols or graphics that represent one thing to 
all learners. For example, checkmarks can be interpreted as incorrect, or mean that something has been 
completed. It is important to set learner expectations at the beginning of assessments when using symbols 
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or graphics as feedback. Learners usually appreciate graphics because they are more likely to remember 
the lesson learned. 

Instructions 

Another important part of online self-assessments is the instructions. Instructions must be clear and 
concise because there is no instructor present to provide clarification of directions or ask to clarify any 
content. Well-written instructions will address both how to use the self-assessment as well as what 
content is contained >vithin the assessment. Instructions should set the learner’s expectations for what will 
take place during the assessment, approximately how long the assessment will take, and explain how and 
when the learner will receive feedback. Instructions should be easily accessible at all times during the 
assessment in case the learner needs to refer to them. 

Placement 

When implementing online self-assessments, they should be placed to test knowledge domains at the 
appropriate time in the learning process. If designers place a self-assessment within the content 
prematurely, this can result in a poor assessment and can be de-motivating to learners. A risk for online 
learning is that learners who aren’t motivated will rarely complete an online course. When a learner is 
well prepared for a self-assessment, the experience may be fun and motivating. 

Additional Tips 

Remember to design for learners who are trying to “cheat” their way through an assessment. In true and 
false assessments, these learners think “true” is most likely the correct choice, for multiple-choice 
assessments the answer choice “C” is the most common correct response, or the longest answer choice is 
often correct. Be sure to test and re-test assessments on many different types of learners in order to make 
it difficult to “guess” correct answers. 

In multiple-choice use consistent patterns such as four answer choices for every question. Make sure all 
answer choices are approximately the same length, and that all answers deal with the same content, and 
test learning objectives. Never offer learners seven different answer choices for one question because th^ 
will get fiiistrated. 

Don’t try to make assessments too fun or cute. Adult learners are motivated for different reasons than 
children. If assessments are well designed and the content is applicable, learners will enjoy the learning 
experience and have fun because of their own motivation. 

Don’t try to “trap” learners. All answer choices should seem like valid responses to the leaner, and never 
test on small details. 



Conclusion 

If designed correctly, on-line self-assessments can provide many beneftts to learners, and can be valuable 
tools for designers who want assurance that learners are achieving learning objectives. However, if 
designed or implemented incorrectly, online self-assessments can be extremely frustrating for learners, 
and serve no use for designers. Learners have strong opinions about what makes a quality online self- 
assessment, that’s why it’s critical to continue testing self-assessments and incorporating learner 
feedback. 
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On-Time and Within Budget: The Project Manager’s Credo 

Project management involves the application of tools and techniques designed to meet a set of objectives 
within a stated time-frame using a given numbo' of resources and dollars. Course developers wiio have 
been in the business of managing the delivery of instructor-lead training for years now are redirecting 
those skills and principles to the realm of e-leaming. This article introduces some additional points for 
consideration — some drawn from lessons learned in IT project management and some drawn from the 
latest developments in the area of learning objects. 

Learning From FaUure 

In his book Virtual Learning, Roger Schank says that “Real thinking never starts until the learner fails.” 
A review of some of the major reasons for e-leaming project failure may help identify some lessons for 
the course developer about to launch a new e-leaming project. Project Managing E-Leaming cites 1 8 
reasons why e-leaming projects fail. The most common reasons centered around 4 major areas: 

1 . Failure maintain customers and project sponsor commitment and involvement 

2. Failure to plan for modularity and reusability in e-leaming courses. 

3. Failure to adequately manage project scope, risks, and evolving requir^ents throughout the e- 
leaming project. 

4. Failure to perform meaningful reviews to ensure an environment of continuous process 
improvement. 

Bullet-proofing an e-leaming course development project requires the developer to put best practices in 
place to avoid these pitfalls. Below are some of the ways of ensuring e-leaming project success. 

Keys to E-Learning Project Success 

E-Leaming project success comes from a wide variety of factors, requiring a heady mix of creativity, 
management prowess, and the application of tools and techniques drawn from best practices in the areas 
of course development, project management, and information technology. These practices can help the e- 
leaming course developer sidestep the 4 failure scenarios described above. 

1. Keep e-leaming customers involved from project launch through final review. Far too often 
customer involvement begins to wane shortly after the initial project launch. Successful projects 
need customers intensely interested throughout the entire project. Some of the techniques used to 
maintain customer involvement include: 

❖ Help them create a vision of the completed e-leaming course during the project launch. Even 
going so far as to have them mock up a “box” for the course, complete with logo, name and 
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primary selling points on the front of the box and a list of key features and objectives on the 
back will go far to create a sense of ownership and excitement about the project. 

*t* Produce the e-leaming course in a series of time-boxed delivay cycles using live or virtual focus 
groups that include the customer and major stakeholders. Use real e-leaming product — ^not 
throwaway prototypes. 

❖ Have at least one member of the customer area included in the ongoing development of the 

course — either as an official manber of the project team or as an ex officio member. Set up a 
virtual project office for collaborative work on the course, review and collection of course 
assets, and storing documentation about the project. Whenever possible, have this customer 
demonstrate the evolving course product at the focus group sessions. 

♦2* Include the customer in project Post-Mortems and interim project reviews and provide ongoing 
feedback. Summarize Post-Mortems from previous projects at project launches to raise 
awareness of best practices and alert current project participants to potential risks based on past 
unexpected events. 

2. Begin thinking in terms oflenming objects, templates, and reusability early in the e-Ieaming 

initiative. Learning assets are too valuable not to organize and utilize to their fullest potential. 

Managing these assets effectively is the secret of successful e-leaming course delivery. Here are 

some recommended practices to build reusability and adaptability into the e-leaming course 

development process. 

♦I* Plan for the future by staying current on e-leaming standards. Every piece of learning content 
should be organized and categorized for ease of reuse and retrieval. Learn about e-leaming 
metadata and how it can help make reuse and sharing of content simpler. 

❖ Define a set of templates to be used for most e-leaming course pages. Having a relatively small 
number of these not only simplifies and expedites course delivery but gives e-leaming content a 
consistent look and feel. Eventually, it will become relatively simple to estimate the effort 
required to deliver a course based on your experience with your repertory of templates. 

❖ Categorize various learning objects and templates by degree of interactivity and complexity to 
make projected estimating more accurate. Each type of asset carries a cost of time and effort to 
develop. Static text-only pages take little time but may be ineffective as teaching tools. Highly 
interactive pages with audio and video components often provide powerful learning experiences 
but can take days or weeks of development time and carry exorbitant production costs. Since 
most courses involve a broad mix of interactivity, having templates and cost estimates 
established in advance can help the course developer come up with better project estimates — 
estimates that can be fine-tuned with ongoing experience in subsequent e-leaming projects. (The 
presentation illustrates various levels of interactivity by demonstrating segments of e-leaming 
courses, ranging from a rather static “page-turner” to an excerpt from a highly sophisticated 
simulation.) 

❖ Explore the potential of using databases to generate course pages. Organizing content as 
described above can ultimately allow for creation of course pages “on the fly,” based on learner 
preferences or interaction. It can also allow the same course content to be delivered with custom 
“branding” for e-leaming customer site. 
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3. Manage scope, risks, and changing requirements using clear-cut statements of work, role 
definitions, and adaptive iterative e-learning product delivery processes. The first two items 
may seem almost too obvious, but short-changing either can lead to e-leaming project disaster. 

Using an iterative product delivery process serves to keep the customer involved as well as provide 
tangible proof of progress toward completing a successful e-leaming product. 

❖ Create detailed statements of work, defined in terms of templates, learning assets. Map each 
item to learner objectives and determine appropriate tradeoffs to balance overall course 
effectiveness to development costs. The customer can assist in this selection process and make 
decisions based on learning needs and budgetary constraints. At this stage, educate the 
customer in concept of tradeoffs as the project progresses and changing requirements are 
identified. 

❖ Use simple tools such as context diagrams (Figure 1) to define roles and responsibilities during 
the life of the e-leaming course development project. 




Figure 1 - Sample Context Diagram 

❖ Use a priority matrix (Figure 2) to show the relative priority of time, cost, and scope/quality. 



Target 


1 


2 


3 


Measurement 


Time 










Cost 










Quality/Scope 










Figure 2 - 


Priority Matrix 



❖ Perform risk assessments for every project; quantify them based on likelihood, severity of 
impact, and degree of control; and take appropriate action to avoid or handle the most 



361 



Shackelford & Aleckson ❖ 377 



threatening. Keep these “top 10 risks” visible to all project stakeholders by reporting them in 
regular weekly progress reports. 

In today’s world, changing requirements are often not only inevitable but desirable. The key is 
to minimize their impact through an iterative delivery process. Break every project into 3 
cycles, with each cycle culminating in the delivery of working e-leaming course product. 
Customers working in this environment soon learn about the trade-offs required to deliver the 
course on time and on budget while at the same time delivering the best possible e-leaming 
product. 

4. Make the project review process an integral part of every e-leaming effort The key to 

continuous process improvemait is continuous process evaluation. The lessons learned from an e- 
leaming project should inform the next project to reinforce successful practices, to prevent the 
repetition of inefficient or erroneous ones, and to anticipate risks based on problems encountered in 
previous projects. Some simple but effective techniques include: 

❖ Schedule formal post project review sessions at the end of every e-leaming project. Survey 
team members and stakeholders in advance to prepare preliminary findings, but plan to use 
these surveys as springboards for discussion at the review session. Capture and summarize the 
results of these sessions and make them part of your virtual project office. 

♦♦♦ Build a risk database to draw upon for future projects. Similarly, build a database of best 
practices. (A wise e-leaming project manager might even view these pieces of data as learning 
assets for ongoing e-leaming and knowledge management activities!) 

Build a Pattern of Success 

The four areas outlined here can help ensure e-leaming success. Sharing of best practices is the obvious 
first step to a successful ongoing e-leaming initiative. Customer involvement, capitalizing on the 
potential of learning objects, sound project management control coupled with adaptive iterative processes, 
and a commitment to continuous process improvement will provide a solid basis for current and future e- 
leaming projects. 
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Introduction 

One question that all designers of Web-based instruction ask themselves is, “Do our online materials meet 
our learners’ needs?” Answering this question requires careful consideration of three factors: (1) who the 
learners are; (2) what their needs might be; and (3) how to design online instructional materials to best 
meet these needs. At the UW-Madison Writing Center, our primary mission is to provide academic 
support for undergraduates who need assistance with writing tasks assigned by their instructors. A major 
component of this academic support is our Online Writing Center, which consists of hundreds of Web 
pages offering writing advice available whenever students need it. Although the Online Writing Center 
has been in existence for several years and has been quite popular, to better serve the needs of our learners 
we were motivated to reevaluate and improve its design. In this paper, we will present and advocate an 
approach to designing task- and learner-centered Web-based instruction that employs principles of 
structured writing to meet the demands of “just-in-time” instruction and performance support. We have 
recently applied this approach in redesigning the Online Writing Center’s Web site 
(http://www.wisc.edu/writing), from which we will show examples that will demonstrate their improved 
pedagogical effectiveness. The concepts and methods involved in our redesign are straightforward yet 
powerfiil, and will help all designers of Web-based instruction better meet their learners’ needs. 

The Importance of Identifying Learners and Their Tasks in Web-Based Instruction 

In any pedagogical situation, properly identifying the target audience can help educators tailor their 
instruction to best convey material to a particular demographic group, and especially so on the Web. In 
our case, the UW-Madison Online Writing Center Web site aims to provide a wealth of writing 
instruction available whenever learners need it; our audience primarily consists of undergraduates 
responding to writing tasks or assignments in their courses, much as in the brick-and-mortar Writing 
Center on campus. Yet in reevaluating our Web-based instructional materials, we felt that they were not 
adequately designed to meet the needs of our target audience. 

Web usability studies can provide educators with valuable insights for improving the design of their 
instructional Web sites. The more we understand how users interact with information on the Web, the 
better we become at designing effective content for online learning. On the whole, usability studies reveal 
one main user tendency particularly relevant for the design of online instructional materials: people tend 
to use the Web in a “purposeful” way that is task-driven and focused on getting answers to specific 
questions, or on obtaining information needed to perform specific tasks and accomplish specific goals. 
For example, users visit our Online Writing Center Web site not because they want to learn everything 
there is to know about citing sources using the American Psychological Association documentation 
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system; th^ come there at 3:00 a.m. the morning before their paper is due, wanting to know how to 
create an entry in a reference list for a particular kind of source — ^nothing more, nothing less. Whether it 
be obtaining information for writing an academic paper, gath^ing data to make a financial decision, 
researching restaurants or movies for evening entertainment, or researching laptop computers for 
purchase, the impetus for going to specific Web sites is largely task-driven. Given the reasons why people 
actually use the Web, the most effective Web sites are those that enable users to get the information they 
need quickly and easily. 

Meeting Users’ Needs with the Just-in-Time / Performance Support Paradigm 

Knowing how users interact with Web-based information can help educators design and write Web 
content better suited to how students use the Web medium. If we look outside academia to corporate 
education and training, one trend is a shift away from content-driven instruction toward task-driven “just- 
in-time” instruction and electronic performance support. Capitalizing on the fact that people are highly 
motivated to acquire new knowledge or skills at they time they need them, a j<iist-m-time instructional 
strategy provides instruction just before the learner needs to apply new knowledge or skills. In contrast, 
traditional content-driven instruction tends to use a “just-in-case” strategy, often providing abundant and 
extraneous content not immediately relevant to the task at hand. The concept of penformamce support 
goes hand-in-hand with such a just-in-time strategy: a performance support application can replace live 
coaches and is extremely task-centered, providing users with Just enough guidance they need to perform a 
task at the very time th^ need it. Unlike in traditional instruction, where the learner gains knowledge 
decontextualized from the actual “doing” of the task for which such training was intended, in a 
performance support environment the learner acquires necessary knowledge while performing the task. A 
well-designed performance support application strives to make the structure of a task explicit such that 
even novices will be able to perform it. Because of people’s task-driven approach to using the Web, a 
just-in-time instructional delivery approach using performance support principles — ^providingyitf / enough 
information Just before users need to apply it — would be well-suited to a Web environment. 

One way we have redesigned the Online Writing Center’s Web site to accommodate a more task-driven, 
performance support format is to rewrite our navigational menus to cater more directly to our learners’ 
tasks. Our online “Writer’s Handbook” consists of Web pages containing a great deal of information 
about writing, yet we felt that this information was not as accessible as it could be. Currently, our Writer’s 
Handbook pages are structured around the content they convey, much as an instructor might present a 
linear series of lessons. However, we felt that to best serve the needs of our target audience, our pages 
should above all respond to the writing tasks that they are asked to perform in their coursework. In other 
words, our pages currently address the “What” of writing, while the needs of our users center more 
around the “How” of writing. For example, note the changes we have made in our APA Parenthetical 
Citations menu (Table 1): 



• Placement 


• Where to place parenthetical citations 


• Two or more authors 


• Cite source with one or two authors 


• No author 


• Cite source with three or more authors 


• Multiple references 


• Cite source with no author 


• Punctuation 


• Cite multiple sources in one reference 


• Electronic sources 


• Cite an electronic source 



Table 1; Cbntait-driven menu (left) vs. Task-driven menu (right) 

The content of the menu links is still self-evident, but now students can better match their tasks with the 
tasks presented in the menus. 
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In educational contexts dealing with relatively stable subject matter, Web designers may be asked to 
create online learning materials that provide just-in-time instruction with just enough information to help 
learners accomplish their necessary tasks. To meet these needs, we feel that Web designers should 
employ design strategies that incorporate elements of just-in-time instruction and performance support. 
These concepts’ emphasis on users’ tasks underlies a more general shift from content-centered to user- 
centered instruction. Instead of focusing on what they know themselves, Web designers could first focus 
on FAQs (what their users need and want to know) and use them as the basis for tiieir content outlines. A 
user-centered designer ideally would discuss a prototype content outline for a Web site with several 
potential users, listen closely to their questions and suggestions, and then incorporate user feedback 
directly into their site design. In this manner, Web designers would be starting from a rigorous user- 
caitered perspective — perhaps the first and most fundamental change Web designers can make to 
improve their Web writing. 

Using Structured Writing to Design User-Centered, Just-in-Time Performance Support 

Shifting toward a user-centered perspective allows Web designers better to understand why learners use 
the Web, but one also needs understand how uscts actually read online material, and to design it 
accordingly. Through numerous usability studies, Jakob Nielsen and others have shown that users tend to 
read Web pages differently than they would a printed text. They consistently: 

1 . Scan page(s): Users look quickly at main titles, headings, menu labels, and headlines to determine 
A\frether to spend more time there. 

2. Skim content: Users selectively search for information they want and ignore the rest. 

3. Read and retrieve: Uso's start to read only after locating the information they want. If documents 
are more than a few pages in length, most users will print them out. 

4. Leave: Users move on quickly after locating the information they want. If the site contains 
interesting information, they may bookmark it for future use. 

We need not look fiirther than our own online behaviors to confirm the above. As the Web is largely used 
for purposeful, task-driven problem-solving and information retrieval, naturally users would read online 
text much diff^ently than they would printed documents. For maximum pedagogical effectiveness, 
designers of online instruction should be aware of Web reading practices when writing for the Web. 

Much as the just-in-time / performance support paradigm has helped us address learners’ task-driven 
needs, Robert Horn’s principles of structured writing has helped us learn to write for the Web in 
accordance with how learners actually read online. Structured writing is a user-centered design model that 
best allows us to meet the task-driven requirements of just-in-time instruction and performance support, 
and in turn best meet the instructional needs of our learners. As Horn insists, structured writing is not 
simply a way of formatting text; it is a way of analyzing a communication and teaching situation. Two 
major components of structured writing involve the conc^t of “information blocks” and the “systanatic 
integration of graphics” into documents. In structured writing, information blocks replace paragraphs in 
terms of content organization and presentation, and consist only of essential, factual information readers 
need to know to perform a certain task; clearly-labeled graphics, such as tables, aid information blocks in 
presenting material quickly and efficiently. 

Below are two examples of how we redesigned existing matanal using these structured writing principles. 
In the first example (Figure 1 ), the old page for “APA Headings” consisted primarily of text in a narrative 
format with extraneous information, while the new page employs information blocks and graphics to 
explain efficiently and visually the complex heading structure the APA system requires. 
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AFAHeatSnta 

5 CENTIMD OTPSRCASK HIJIDINC 

1 Caatared Qppareaaa and Lovareesa Bead too 

2 Centarad# XtaJiciaacf, Ojpaecaaa aad Z/Owarcaaa Haadincr 
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Figure 1: Old APA Headings Page vs. APA Headings Page with Structured Writing 

In the second example (Figure 2), tables with column and row headings immediately indicate the type of 
content presented, while the visual format allows the user to quickly scan to find the necessary 
information. In their existing format, our examples of how to create parenthetical citations using the APA 
system are displayed on a single, long scrolling page, which makes locating a needed example more 
complicated than necessaiy. 
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Figure 2: Old APA Parenthetical Citations Page vs. APA Parenthetical Citations Page with Tables 

Conclusion 

We recognize that performance support and structured writing are not suitable for all written teaching and 
training materials. For example, if we were trying to persuade users or students of a particular argument 
or point of view, the conventions of standard prose may be much better suited. The same may be true 
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when we are introducing a complex new topic, or when simple, discrete instructions are too reductive for 
a given task. 

Yet when there are learning activities in an online environment that involve mastering a process, and 
when the context is just-in-time, where users arrive at a Web site needing to learn how to do something 
right away, these methods of designing instructional material seem ideal — and our extensive user testing 
confirms that. We would urge everyone developing online learning materials to reexamine their online 
content for parts that match the Just-in-time learning situation, and to follow these principles to better 
serve their learners’ actual, task-driven needs. 
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Increasing the Quality and Efficiency 
of Web-Based Course Production 

Jared Stein 

Instructional Designer 

Distance Education, Utah Valley State College 

Introduction 

In February of 2002 Utah Valley State College’s (UVSC) Distance Education development team began to 
formulate solutions to certain problems that had amassed since the department had taken on the 
responsibility of directing and producing web-based distance education courses: high demand with an 
emphasis on timeliness from academic departments and college administration, and our slow, 
inconsistent, and often sub-par production. 

The aim of this paper is to summarize the development of the template-based web production procedure 
currently used by UVSC’s department of Distance Education in an effort to streamline production, 
establish consistency, and improve quality. Besides relating the problems and ideas that sparked the 
development process, I will present an overview of the current process in which our student programmers 
bring a web-based course from edited content to final review, which is now being used in the production 
of more than 150 distance education courses. 



The Problems 

Distance Education at UVSC has experienced a tremendous growth over the past two years. Aside from 
producing new television broadcast courses each year and broadcasting numwous live interactive courses 
each semester over fiber optic lines throughout the state of Utah, Distance Education has begun to 
produce completely and partially online synchronous and asynchronous web-based courses. In 2001 our 
department’s small staff projected we could produce four new web-based courses per semester; we were 
in fact called on to produce an average of 20 new web-based courses each semester with no significant 
increase in staff. 

Because the demand for courses per semester exceeded department expectations and because no 
standards, specifications, guidelines, or rigid procedures existed for web-based course production at 
UVSC, the quality of the courses produced and the efficiency of production was not at the level we had 
hoped for. From the beginning we worked with a small team of student programmers under the direction 
of a full-time developer and designer. These often inexperienced programmers were entrusted with course 
content, the responsibility to turn electronic or hard copy text into web pages, and to provide 
individualized interactive or multimedia components. 

The original “process” consisted of importing a course’s textual content into Macromedia Dreamweaver 
to be coded. In the early stages the programming team foimd themselves so pressed for time that they 
would encourage professors to save their content in Word or WordPerfect as HTML documents, 
eliminating a step in the process. Even the programmers themselves got into the practice of simply saving 
original word processor documents as HTML. This solved some of the time problems but increased the 
variability of course appearance, structural design, and coding. These annoyances could be overcome 
with strict procedure and guidelines, but the other major problem with creating web pages through a 
word processor is the resulting bloated, inconsistent, unstable, and virtually unnavigable code. Web 
pages that are manually coded, or that use advanced web design software (we use Macromedia 
Dreamweaver or HomeSite) are much cleaner and use HTML more efficiently. This makes revising, 
editing, and extracting simpler, and gives the HTML designer more control. 
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It would be unrealistic to expect our student programmers to plan out and hand code all the web pages 
that need to be prepared in a single semester and have them done on time. Creating web pages in Word 
was fast enough to let us produce 20+ courses a semester, which helped us overlook Word’s other 
shortcomings for the time being. But these bothersome aspects of Word-produced HTML documents 
reappeared as we found ourselves trying to revise and edit courses from previous semesters. 

Additionally, because no standards of file naming or folder structure were in place at the time of 
production, courses were often disorganized and counterintuitive. This made finding and identifying files 
unnecessarily time consuming. We quickly decided that a set of file naming conventions and folder 
structure standards were needed so that any programmer in our department could work on one course as 
easily as another without consulting the original programmer. 

Another problem that we confronted was the inconsistency of course appearances. Since most of the 
documents came straight from the faculty to our staff of technical editors to the programmers there was 
little in the way of aesthetic or instructional design. Any design that was in practice came usually as an 
afterthought, or as an independent urge from a faculty or department member. We decided that, in order 
to better represent Distance Education and UVSC through our web-based courses, certain standards of 
appearance must be implemented across all web sites. A consistent design would connect course pages to 
all other pages within a course, would connect courses to all other courses within a department, and 
would connect departments to all other departments offering web-based courses through Distance 
Education. 

To solve these problems we focused on the following questions: 1 ) How can we move away from using 
automatic web page conversion tools (e.g.. Word) and yet meet our quotas and deadlines? 2) How can we 
maintain a standard of design and a consistency of appearance across courses without impeding 
production? 3) How can we facilitate future revising and editing of web-based course content? We 
answered these questions with a combination of a stable HTML template, a simple, flexible CSS, and set 
of guidelines and procedures for graphic design, file and folder structure creation, and overall course 
production. 



Developing a Template, Creating a Process 

It became immediately apparent that to streamline our production of web-based courses we needed a 
fairly rigid procedure supplemented by guidelines for our student programmers to follow. The procedure 
followed from our experience with programming courses, attending to the need for more precise 
directions and structural specifications. The guidelines and visual specifications began with meetings 
between the departments instructional designers, who initiated a list of formatting and design components 
that should be consistent across courses. Knowing that much of a web site’s appearance could be 
controlled by cascading style sheets, we separated our specifications into elements that could be 
maintained and controlled in the CSS and those that had to be attended to by the student programmer. As 
we experimented with a master style sheet for our burgeoning prototype template, the list of elements that 
would have been controlled by the students grew shorter, eliminating consumptive editing and design 
steps in the programming process while producing the same visual effect across courses. We also listed 
items in the style sheet (and in what would be the template) that would need to be changed from course to 
course or from department to department. 

Once we had determined what our formatting and design standards for the course were, I sat down with 
the senior developer to establish file and folder structures and naming conventions that would be 
generally consistent. We determined which pages and files were required in all courses and which pages 
and files were optional or variable. With this master list I produced the first version of our HTML 
template, a simple framework designed in accordance to our instructional design standards and following 
our file and folder naming conventions and structures. 
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The idea behind the template was that a few basic pages could be copied, renamed, and then modified to 
preserve design and common elements. Our prototype template, based on this idea, was designed so that 
common elements that would appear on every page throughout a course could be changed in one step 
using Dreamweaver’s or HomeSite’s extended find and replace commands. The body of the web page 
was designed to hold content without a lot of code to allow our student programmers to copy and paste 
text content from a word processor, add a few formatting tags (e.g., adding paragraph tags or line breaks, 
bolding or italicizing text), and save the new page. All other major formatting and design would be 
contained above or below the content, inherent in the template or style sheet. 

With a file and folder structure reference sheet and a HTML specifications and standards reference sheet 
we brought the prototype template to our programmers and incorporated the web production procedure 
into our overall course production process. 

Results of Our Move to Template-Based Production 

Currently we are tracking student programmer production of web-based courses using our new template, 
style sheet, and procedure guidelines. Most obvious is the overall improvement of web-based course 
layout and design quality, though the most significant element may be the underlying consistency and 
stability of the template-based HTML and CSS that runs through all of our courses. Efficiency too has 
been increased, as new course production programming time fell from an average of 30 hours per course 
in Summer 2001 to a preliminary projected average of 20 hours per course in Summer 2002. We will 
continue monitoring and tracking programming times for content portions of our web-based courses 
throughout the Summer and Fall of 2002. 

Additionally, we will attend to the possibilities of further automatizing and streamlining the process to 
bring greater production efficiency and design quality. One unforseen positive of our template process 
and the establishment of a stable production procedure is the increase in portability of our web content. 
No longer is our content stuck within WebCT; since our textual content pages are produced completely 
outside of the WebCT we are able to preserve our content in a form that is easily adaptable to other web 
course platforms. The future of our Internet course development will likely include the use of XML and 
XSL templates for complete portability across web course development platforms, SCORM compliance, 
and greater integration with college network databases. 
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' An example of a finished template-based course, COMM I SOO, as it appears within WebCT. . . 

Biographical Sketch 
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experience in the design and delivery of distance education over the Internet. His work for the 
departments of Distance Education at both Utah Valley State College and Utah State University has 
taught him to conquer problems and tame unruly hardware, software, and design elements for more 
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Ceramic Institute. 
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Creating Course Evaluations for E-Leaming 

Renee Stoll 
Training Manager 
TDS TELECOM 



Sandy Ellis-Brye 
Supervisor of E-Leaming 
TDS TELECOM 

Introduction 

E-Leaming, Online Learning, Distance Education, Web-based Training, Computer-based Training, the 
list goes on and on. Companies today are taking a hard look at alternative methods of delivering training 
to their employees. TDS TELECOM is no exception. With an employee base of close to 3500 spread 
across 29 states throughout the US, the TDS TELECOM Training Team is continually challenged to 
provide quality training, just in time, in the right amount, targeted to the audience — all without breaking 
the bank. 

TDS TELECOM has embraced this challenge enthusiastically by exploring many alternative delivery 
methods including: 

❖ Audio teleconference 

❖ Web conferencing 

❖ Video conferencing 

❖ Computer-based training (CBT) 

❖ Web-based training (WBT) 

The current focus is on developing WBT in-house, which is the focus of this paper. From anecdotal 
comments, initial participant reaction to the WBT programs was favorable, however, the Training Team 
wanted to better quantify the effectiveness of their e-leaming. The drivers behind this desire to evaluate 
their e-leaming are listed below. 

Driver: Improve Instructional Design 

The Instmctional Design team consists of experienced classroom trainers and designers and multimedia 
designers. Even so, web-based training is still too new to have tried and tme methods of instmctional 
design. Over the past 3 years, the team had made significant improvement in the design of their WBT 
programs. Evaluation data would help this team understand, from the participants’ point of view, what 
was helpful, what was annoying and what was unnecessary. 

Driver: Justify Development Costs 

TDS TELECOM made the decision to develop the skills and knowledge to develop WBT in-house. Costs 
associated with this decision included instmctional developers time, software development tools, outside 
consultant fees and IT resources. In order to continue down this path, the Training Team needed to assure 
themselves and upper management that e-leaming was delivering effective training. 
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Driver: Gain Support for Continued Exploration 

In addition to justifying the costs to date, the Training Team wanted the support to continue to look for 
creative solutions and build their skills. They knew that in order to gain that future support, they needed 
evaluation data to back up their investments. 

Driver: Ensure Training Effectiveness 

The most important driver of this project was the on-going need to ensure training effectiveness. The 
Training Team prides itself on focusing on improved performance and needed the proof that their efforts 
were making an impact. 



Evaluation Research 

Once deciding to invest the time and effort into evaluating their programs, the Training Team had to come 
to consensus on an evaluation methodology. Numerous members of the team had past experience with 
Kirkpatrick’s Four Levels of Evaluation and after comparing this model with the team evaluation 
objectives, they decided to use this as the base of their process. A brief overview of the Four Levels of 
Evaluation as developed by Kirkpatrick follows. 

Level One: Evaluating Reaction 

Reaction is measured immediately after the completion of the training session and is designed to measure 
how the participants felt about the program. This data can communicate how well the audience liked the 
training, the instructor, the delivery methods or other factors. It is designed to provide the trainer and 
instructional developer with feedback on the course itself, which can then be used to improve the 
program. 

Level Two: Evaluating Learning 

Learning is measured immediately before and immediately after (or during) the training session. The pre- 
assessment is designed to capture what the participants know going into the program. The post- 
assessment can then show what they’ve learned as a result of the training. It is designed to provide the 
trainer and instructional developer with data regarding the effectiveness of the instructional method and 
activities. 

Level Three: Evaluating Behavior 

Behavior is measured after a determined amount of time after the session. The purpose of this evaluation 
is to measure whether the behavior has changed back on the job. This could include a pre and post 
assessment, use a control group, or relay on operational measures to make the comparison. Analyzing this 
data requires looking at Level Two results and the work environment. 

Level Four: Evaluating Results 

Results refer to the impact the training has had on the organization as a whole. Did Sales increase? 
Customer Service ratings? Efficiencies? This is the most challenging level to evaluate because increased 
results are typically not solely the result of effective training. Most often, a number of organizational 
factors contribute to the successful results. 
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Developing the Evaluation 



The Training Team had been using Level One and Level Three evaluations for their Instructor-led face to 
face classes for a couple of years. The initial idea then was to simply modify the existing evaluations and 
call it a day. As the team took a closer look, however, they realized they would need to think a bit more 
about their objectives for the evaluations before simply converting the existing form. That became the 
starting question, “What is it we want to know?” 

After a great deal of discussion on this question, however, the team realized the root question was, “What 
will we do with the data?” Answering the use question then pointed them in the right direction for 
evaluation questions and content. 

The result of these discussions was to focus on Level One evaluation first for two reasons; 1) it would be 
the easiest to create, distribute, collate and analyze, and 2) it would give immediate feedback on the 
instructional design. Their were a number of additional WBT programs on the horizon and the team 
wanted to make design modifications that could be put in place with the upcoming programs. An 
evaluation form was developed and a process defined for capturing participant responses. The main 
components of the new form are listed below. 

❖ Program Relevance and Timeliness 

❖ Content and Practice Exercises 

❖ Network and Computer Issues 

❖ E-Leaming Environment 

Implementing the New Evaluation Form 

The implementation of the new evaluation form has evolved over time. The first distribution was through 
email using MS Word. Evaluation forms were sent to participants after the training. Participants then sent 
their completed form back through email and the results were manually entered into a database. The 
process was time consuming and didn’t allow for anonymous feedback, which may or may not have 
affected results. Even so, this was better than no feedback at all. 

The second phase linked the MS Word file to the WBT program. This improved the response rate, but 
was still very cumbersome as the participants had to save the file, then attach it to an email and the results 
were still manually entered into the database. 

At this point, the team solicited the help of the IT department and designed an evaluation using the same 
development tools they used to create the WBT program. The evaluation then matched the look and feel 
of the rest of the program and appeared as the final stage of the training as opposed to an annoying add- 
on. The results were dumped into a database, which eliminated the manual data entry and allowed for 
anonymous feedback. 

The additional of Level One evaluation to TDS TELECOM’S WBT programs has brought the team one 
step closer to their initial drivers. 

❖ Improve Instructional Design 

❖ Justify Development Costs 

❖ Gain Support for Continued Exploration 

❖ Ensure Training Effectiveness 
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There is still much work to do and other levels to evaluate, but this start has given the team the support 
and backing th^ need to continue down their e-leaming path. 

Recommendations 

❖ If you’re not already familiar with the different levels of evaluation, do some research. It’s 
important that your organization agree on a methodology whether it be Kirkpatrick’s or 
someone else’s. This is key to ensuring understanding of the results. 

❖ Take some time to determine your drivers. Ask yourself why you want to evaluate your 
training. The end process will differ depending on the drivers you determine. 

❖ Next, determine how you want to use the results. If you’re not going to use the data, don’t 
capture it - you’ll just be wasting your participant’s time. This discussion will lead you into 
what to ask, how to ask it and when to ask. 

❖ Get your IT team involved early to help you figure out the technical issues and options. They 
can consult on the mechanics of the implementation of your evaluation process. 

❖ Don’t give up. If at first the response rates aren’t what you had hoped, remind yourself that 
some is better than none. It may take your participants a while to realize you REALLY want 
their input and are REALLY going to use their feedback. 

Summary 

E-Leaming at TDS TELECOM is a continually evolving practice. The Training Team continually 
researches new tools, practices and methods for providing training using alternative delivery methods. As 
their delivery methods evolve, so too, will their evaluation processes. Future goals of the team include 
increasing response rate, building a database to capture the evaluation results, publishing reports on the 
results of the evaluation data and exploring Levels Two, Three and Four. As we continue to see articles 
like Kaliym Islam’s “Is E-Leaming Flotmdering?” published in the May 2002 issue of e-leaming 
magazine, it is clear that we all need to take a close look at evaluating our programs and showing the 
benefit th^ are having on employee performance. 
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Introduction to Plagiarism: The Prevalence of Cheating 

Several research studies examine the reasons students cheat. The studies of plagiarism among college 
students are many, and the evidence presented in these studies typically suggests that plagiarism is 
rampant. These studies would estimate that 40 to 60 percent of students cheat during their college careers. 
Some go lower: in “Plagiarism is Rampant, A Survey Finds,” the New York Times reports in 1990 that 
researcher Jerold Hale called plagiarism “an epidemic hurting 30 percent of the student population across 
the country.” Some researchers go higher: Charlotte Allen, writing in 1994, attested, “Most studies of 
college cheating indicate that the percentage of cribbers has not changed much over the past twenty or 
thirty years: About 60 to 80 percent of college students admit, as they did during the Sfacties and 
Seventies, to having fudged an exam, term paper, or problem set at least once during their four years.” 
Donald L. McCabe and Sally Cole (1995), the president and executive director, respectively, of the 
Center for Academic Integrity, reported ttot “more than half’ of the MIT students surveyed in 1992 
“acknowledged” that they had engaged in unsanctioned “collaboration.” 

Less is known about graduate student behavior. One study (Brown, 1995) found that graduate business 
students believed they were more ethical in their academic behavior than were their undergraduate 
counterparts. Brown reported, however, a similar incidence of academic dishonesty — ^as well as similar 
explanations for their behavior — among the graduate and undergraduate students in the study. In another 
study of graduate students. Love, et al. (1997) discussed the need to consider undergraduate behavior 
when investigating graduate student behavior. The prevalence of plagiarism in undergraduate education 
may well affect the graduate education arena. The authors mentioned a study by GreCTe and Saxe (1992) 
that found 81% of the undergraduates they surveyed had cheated at least once. Of these, 77% indicated 
th^ intended to pursue graduate education. As a result, behavior as an undergraduate is probably 
relevant to behavior as a graduate student. 

A problem with all these studies is that they generally do not differentiate categories of “cheating.” They 
may specify that “cheating” includes sneaking answers into exams and plagiarizing. But what do they 
mean by “plagiarizing”? Do they only mean submitting papers written by others? Or do th^ include 
copying passages without attribution? Do they mean failing to cite sources of ideas? Do they mean 
patchwriting? Such specifications are seldom offered. So, when we look to these studies to learn how 
prevalent plagiarism is, we have to know in advance what we ourselves mean by “plagiarism.” Also, we 
have to be alert for cues as to what the researchers mean by it; whether th^ include plagiarism in their 
category of “cheating” or “academic dishonesty”; and whether th^ communicated these distinctions to 
research participants whom th^ were surveying. 

Examples of “plagiarism” and “cheating” defmitions in reports of research may illuminate the problem. 
Doris Dant (1986) made clear that she includes copying papers and copying passages of source texts 
\s4ien she said that although 94% of the high school students she surveyed reported having received 



^ Stover* Kelly ❖ 397 

ERIC 



379 



instruction in attribution of sources, 80% reported having “copied some to most of their reports.” But 
consider how this does — or doesn’t — compare to Richard Pass’s findings in the same year. Reporting a 
decline in ethical behavior that he dated to the 1960s, Pass pointed to two surveys. In one, “approximately 
40 percent of the university students surveyed considered cheating to be a normal part of life, and 30 
percent felt no guilt about cheating in college.” In the other, “approximately 75 percent of the high school 
students participating in a recent California Department of Education survey admitted to cheating on tests, 
and more than 42 percent believed that there were sometimes good reasons to do so.” But then we learn 
what Pass meant by “cheating”: his category includes writing tutors’ assistance in the “style and structure 
of a paper”! It is hard to determine from all these studies whether academic dishonesty actually is on the 
rise; and it is impossible to determine A^ether plagiarism is — precisely because so many disparate 
activities are variously included in the category of plagiarism. 

But when the studies specify their categories, th^ provide some illuminating snapshots. Christopher 
Hawl^ (1984), for example, reported that the Bureau of Applied Social Research conducted a 1964 
surv^ of 425 undergraduate students at a Midwestern university. “Piffy-one respondents (12.0%) 
reported that th^ had specifically asked another student or some other person to write a paper for them 
while at college and sixty-two students (14.6%) indicated that they had actually handed in a paper written 
by a colleague.” Pollow-up research to the 1964 would determine whether we have a greater problem 
today. Most faculty belieE'fear/suspicion is that we do, especially because of the Internet. However, 
Lunsford and Ede (1994) cautioned that the pedagogical use of testing norms — and the culture of 
individualism that underlies it — can bring teachers to a “near obsession” about plagiarism. 

The few longitudinal studies do indicate a rise in the incidence of plagiarism. Schab (1980) reported a 
study conducted with 1,000 Georgia high school students in 1969 and again in 1979. “A significant 
increase in plagiarism was revealed, as indicated by the change from an admission in 1969 of 63.5 percent 
of those going to college to 78.7 percent in 1979. Among the others, the 1969 group’s figure was 70. 1 
percent while in 1979 it had increased to 81.1 percent.” 

Reasons for Cheatii^ 

One stu(fy examines the correlation between procrastination and plagiarism (Roig & DeTommaso, 1995). 
The authors found that students who scored high on the plagiarism scale showed a main effect for 
procrastination. Although not statistically significant, those who scored high on procrastination also 
scored high on plagiarism. Roig and DeTommaso believed grade, time, and task pressure play an 
important role in cheating. These factors seem to play a significant role in a student’s perception that 
cheating is the only acceptable alternative to lessen the stress associated with the above factors. 

Love et al. (1997) found that the professors play a role in the likelihood of cheating where “the leniency 
of professors and a tendency to avoid addressing issues of cheating and plagiarism were seen as factors 
contributing to cheating and plagiarism among graduate students.” Simmons (1999) added ways in which 
teachers support plagiarism, including the fact that in large classes it is unlikely to be recognized. 

Howard (1993, 1995) identified patchwriting as a type of plagiarism. Students patchwrite when trying to 
learn unfamiliar discourse. However, because most people patchwrite, Howard (1993, 1995) 
recommended that patchwriting not be considered a form of plagiarism and academic dishonesty. Later 
(2000), she recommended discarding the term plagiarism altogether, and instead specifying the textual 
behaviors (fraud, patchwriting, failure to cite, failure to provide quotation marks) traditionally covered by 
the term. 

We may also consider cultural beliefs. One of these is that the plagiarist suffers from a moral disease. As 
early as 1936, Loomis et al. referred to plagiarism as a sort of “kleptomania.” In 1989, Mallon refered to 
it as a “virus.” And Geosits and Kirk (1983) and Nienhus (1989) searched for a “cure.” Another 
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recognizable cultural theme associates plagiarism with sexual transgression; the plagiarist is an adulterer 
or rapist (Howard, 2000). 

Intellectual historian Ong (1982) asked why plagiarism matters to a culture. When print was introduced to 
Western civilization, he said, exclusive printing rights were granted. “The drift in human consciousness 
toward greater individualism had been served well by print. Of course, words were not quite private 
property. They were still shared property to a degree.” Rhetorician Clark ( 1 994) specified what this can 
mean for pedagogy: “Concern with plagiarism in writing center instruction reflects a pervasive cultural 
concern with intellectual property rights that has gained particular prominence in academia.” 

Where does all this leave the educator? First, it suggests that we should strive to identify just what is at 
stake when we believe we are facing plagiarism. Has the entire paper been downloaded or borrowed from 
someone else? Has the student patchwritten from a source? Second, it suggests that plagiarism matters 
very much to contemporary academic culture. Third, the textual behaviors we gather together under the 
heading of “plagiarism” may actually be occurring more frequently today than they were in earlier 
decades. Fourth, the types of assignments we give and the conditions under which we give them may 
affect the likelihood of students’ resorting to outlawed textual strategies. Fifth, our reactions to the 
incidence of these outlawed textual strategies may evoke in us some powerful emotional responses, in 
which students’ writing becomes mixed up with our own fear of contamination, and deviance. 

Institutional Strategies to Prevent Plagiarism: A Case Study from UMUC 

The University of Maryland University College has developed a two-pronged approach to deal with 
issues of plagiarism. The first is to have an effective policy that clearly defines plagiarism; provides 
specific procedures for students, faculty, and staff to follow; and details the penalties for plagiarism. The 
second approach is to educate students and faculty on how to recognize and avoid plagiarism. 

UlVfUC Plagiarism Policies and Procedures 

University of Maryland University College’s official policy concerning academic dishonesty and 
plagiarism is posted at http://www.umuc.edu/policy/policy_files/ml5025.html. This policy was recently 
reviewed by the university community and updated. The main points of the policy are: 

❖ Plagiarism is the presentation of another person’s ideas or products as one's own. Intent is not a 
factor. 

❖ The burden of proof rests with the faculty member. The faculty member must show the 
plagiarized documents. Without evidence, a student cannot be accused of plagiarism. 

❖ When plagiarism is documented, the faculty member notifies his/her supervisor. After 
discussion with the administrator, the faculty member discusses the alleged plagiarism with the 
student. The student may not withdraw from the course. 

❖ If the student admits he/she plagiarized, the faculty member may adjust the grade for that 
portion of the assignment, the entire assignment, or give the student an F for the course. 

♦♦♦ The faculty member notifies the dean who sends a letter to the student describing the act of 
dishonesty including a description of the agreed upon sanction. The dean sends copies of the 
letter to the Chief Academic Officer and Provost, and to the faculty member. The provost may 
approve suspension or expulsion from the university. 

❖ The student may appeal to the provost if the student believes the procedures were not followed. 
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Educational Opportunities on How to Avoid Plagiarism 

UMUC’s second approach is to provide education opportunities for both students and faculty to increase 
awareness of plagiarism and how to avoid it. The institution looks for ways to make the topic visible for 
both students and faculty. Below are some examples of such initiatives. 

Faculty Focus: Workshops; LIBS 150, Support; and tumitin.com 

UMUC has several strategies for helping faculty avoid, identify, and as necessary, detect plagiarism. As 
early as 1995, the University offered a series of face-to-face workshops on designing plagiarism-resistant 
assignments. This initiative was eventually expanded, in 1997, by offering the workshop online to serve 
the entire, worldwide faculty population. In 1999, a workshop dedicated to topics concerning writing, 
research, and plagiarism was successfully developed. Over 200 faculty have participated in the two 
workshops that have been offered. In spring 2001, at the General Faculty Meeting, strategies were 
reviewed for how to avoid plagiarism, especially through deliberate design of writing assignments. Over 
300 faculty members were present. 

UMUC faculty may now participate in the new, required, online course for undergraduates pursuing a 
major at UMUC. In an effort to assist faculty to update their research skills, one of the library faculty is 
offa-ing a faculty section of the course, LIBS 150, so faculty may become familiar with it, and offer Aeir 
feedback on its content and purpose. Interest in this section has been phenomenal. The course closed in 
one week, it was capped at 100 participants. All UMUC Faculty worldwide also have access to 
“tumitin.com” a plagiarism detection service to which UMUC subscribes. With permission of students, 
faculty may submit student essays to check for plagiarism to help find and prevent plagiarism. UMUC 
provides faculty members with guidelines. 

UMUC supports a full-time director for the Effective Writing Program whose primary responsibility is to 
assist faculty in all curricula in creating and evaluating effective writing assignments. The director 
conducts workshops for faculty. The Effective Writing Program’s comprehensive web site, 
http://www.umuc.edu/ugp/ewp/intro.html, provides information on how to help students avoid 
plagiarism. The site also has materials to help students manage research materials and document sources. 
The CentCT for Intellectual Property, at http://www.umuc.edu/distance/cip/links_plagiarism.html provides 
resources about plagiarism and potential responses to the problem is currently conducting research on 
plagiarism policies at institutions across the United States to identify best practices and provide further 
information to students and faculty on how to address academic integrity issues. 

Student Focus: LIBS 150, Online Writing Center, Newsletter, the Virtual Academic Integrity Lab 

UMUC also offers undergraduates help in avoiding plagiarism. All students must take a library skills 
course, LIBS 150: Information Literacy and Research Methods, within the first 15 credits of their career 
at UMUC. This course introduces online research methods and documentation. UMUC's Online Writing 
Center offers a self-help interactive tutorial to help students recognize plagiarism and ways to avoid it. 
http.VAvww. umuc. edu/ugp/ewp_writingcenter/modules/modules. htmL The tutorial. How to Avoid 
Plagiarism, provides a posttest with certificate to be presented to faculty to show successful completion 
of the module. The Online Writing Center also has links to other online writing resources. 

The Office of Library Services offers an online guide to proper citation format that can help students learn 
when to cite and how to do so propa-ly. The Center for Intellectual Property provides resources about 
plagiarism and potential responses to the problem. UMUC has its own Online Guide to Writing and 
Research, wiiich covers plagiarism concerns. Links to the Writing Guide are in all online courses as well 
as through the Online Writing Center. Put it in Writing (http://www.ad.umuc.edu/docs/G54-01 .html) is 
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developed primarily for our students in Asia but may be used by all UMUC students to help them 
understand proper writing technique. In spring 2002, News @ UMUC, a bimonthly newsletter for 
undergraduate students, focused on plagiarism with several articles on how and why to avoid to it. 

Real people are also available to help online students with their research and documentation concerns. 

The Online Writing Center has writing advisors who will review student essays and provide feedback and 
help in writing. The reference librarians at UMUC’s Office of Library Services are available by e-mail 
and telephone to assist both students and faculty with research concerns. 
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The Collaborative Nursing Program 

The Collaborative Nursing Program (CNP) is a joint effort of the nursing programs at five University of 
Wisconsin campuses - UW-Eau Claire, UW-Green Bay, UW-Madison, UW-Milwaukee, and UW- 
Oshkosh - with support and facilitation provided by UW-Extension. One of the University of Wisconsin 
System’s first collaborative distance education programs, the CNP has been available to Registered 
Nurses since spring semester 1996. Courses have been offered by audiographics, compressed video, 
public television, WisLine’ telephone discussion sections, and, since spring semester 1998, the Internet - 
making it possible for nurses to remain in their communities and on the job while completing a 
baccalaureate degree. 

CNP students affiliate with one of the five UW institutions, which becomes their home institution, where 
they are admitted, register for classes, apply for financial aid, receive advising and other student services, 
and ultimately are awarded their degree. As of May 2002, 215 nurses had graduated from this program. 
Over 200 students are enrolled in one or more nursing courses in any given semester, with about 300 
enrolled in the program, some taking only general education courses. Students are predominately (96%) 
white and predominately (92%) female, highly motivated, and geographically bound. Most (69%) CNP 
students are between the ages of 35 and 54, and work full time in nursing (72%). Most (45%) are 
employed in acute care settings. Seventy percent of nurses enrolled in the CNP have Associate Degrees 
in Nursing (ADN), with most others having nursing diplomas. 

The Study 

During the fall and spring semesters 2001-02, the Learning through Evaluation, Adaptation, & 
Dissemination (LEAD) Center of the University of Wisconsin-Madison was contracted to evaluate the 
CNP. The study was supported by a U.S. Department of Health and Human Services, Division of 
Nursing grant to UW-Madison’s School of Nursing. The study was intended to determine students’: 

❖ Views of attainment of academic goals 

❖ Assessment of the learning environment 

❖ Assessment of their experience with the technology 

♦> Opinions and assessment of educational support services 

❖ Professional and personal outcomes resulting from their educational experience. 



' WisLine is a telephone conference service available to Wisconsin educational, governmental, and non-profit 
groups. 
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The LEAD Center conducted structured telephone interviews with ten CNP students during the 
fall 2001 . Interview findings shaped the development of a survey instrument that was mailed during the 
late spring 2002 to all CNP students and graduates in the program’s database - 535 nurses. 253 usable 
responses were returned, for a response rate of 47%. Respondents fell into the following categories: 



❖ 


Graduates 


91 


36% 


❖ 


Currently enrolled 


125 


49% 


❖ 


Not yet enrolled, plan to 
take courses in the future 


24 


10% 


❖ 


Withdrew fi-om program 


13 


5% 



Findings 

In each of the five areas of the study, findings were positive suggesting this program is effectively 
meeting learner needs and that students are well satisfied with the program, its faculty, support services, 
and the technologies used to make learning accessible to working nurses. 

Attainment of Academic Goals 

Sixty-seven percent of respondents said that the the CNP has made it possible for them to begin working 
toward the BSN degree sooner that expected. Before enrolling in the CNP, respondents had considered 
returning to school to complete a BSN for an average of 4.67 years (range is 0 to 30, standard deviation 
5.16). 

When asked to identify the top two reasons for enrolling in a BSN completion program, 77% identified 
“to prepare for future career advancement opportunities.” Sixty-six percent reported they enrolled in a 
BSN completion program for “personal enrichment or satisfaction. ” 

Learners chose a distance education program (CNP) because it made studying more convenient to their 
work schedule (87%), more convenient to their personal schedules (73%) and they thought th^ would 
save time by not having to travel (51%). Another 42% of respondents said it was simply too far to travel 
to a campus for classes. If a distance education program wasn’t available, 61% of respondents report they 
would not have enrolled in a campus-based program. 

Learning Environment and Technology 

Since all CNP courses are offered via technology, questions about the learning environment were linked 
to the technologies used. Analysis of responses was split into those who had taken Internet courses and 
those who were had taken courses offered by other technologies. Internet students were slightly more 
satisfied than those using other technologies. 

CNP Learners who were “satisfied” or ‘S^ery satisfied” 

Internet Courses Other technologies 

Instructor’s responsiveness 89% 88% 

Access to instructor 91% 87% 

Quality of instructors 94% 87% 

Quality of courses 94% 90% 



404 ❖ The Collaborative Nursing Program 



386 



Educational Support Services 



In examining educational support services, responses were again divided into those who took Internet 
courses and those whose CNP courses were delivered by some other technology. Among those who 
indicated they were satisfied or very satisfied, Internet students again report a slightly higher rate of 
satisfaction in most categories. 





Internet 


Other technologies 


Quality of advising 


85% 


87% 


Access to advising 


85% 


77% 


Advisor’s responsiveness 


85% 


86% 


Quality of technical support 


81% 


71% 


Availability of tech support 


80% 


74% 


Quality of library services 


82% 


67% 


Availability of library services 


81% 


69% 


Orientation to CNP 


87% 


84% 


Orientation to distance ed 


82% 


74% 



Career Impacts 

Respondents were asked whether their experience in the CNP had affected their careers in any of the 
following ways: 

Promotion 

Increased responsibilities 
Change in employer 
Increased salary 
Change in area of practice 
Greater satisfaction with nursing career 

1999 Study 

An earlier (1999) study of the CNP, completed by the Wisconsin Survey Research Laboratory ^ explored 
four broad features of the program: 



8 % 

13% 

14% 

15% 

17% 

69% 



Issues of cost and access 

❖ Technologies and learning modalities 

❖ Availability and quality of support services 

❖ Impact of the program on nurses’ careers. 

This study demonstrated a clear preference for Internet technology and substantially increased 
rates of access to the Internet among Wisconsin nurses; almost 85% now had Internet access at home or 
work, contrasted with only 43% in a 1997 study. Study findings informed the fall 2000 decision of the 
Steering Committee^ to go to an all-Intemet program. Other distance education delivery modes are being 
phased out , with the last audiographics course slated for spring semester 2004. 



^ The Wisconsin Survey Research Laboratory, a unit in the University of Wisconsin-Extension’s Division of 
Continuing Education Extension, closed in May 2000. 

^ The CNP Steering Committee is comprised of feculty/administrative representatives from each of the five UW 
institutions with nursing programs and a representative from UW-Extension. 
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Comparii^ the Two Studies 



Populations 

The response rate for the 2002 study was lower (47%) than the 1999 study (60%). The CNP has also 
experienced growth during the three years between studies - 387 vs 535 students. 

There are two noticeable differences in the populations of the two studies: 

1999 

Actual graduates 60 

Grads as %age of total population 16% 

Had taken an Internet Class 5 1% of respondents 

Expectations and Findings 

When we undertook the 1999 study, we expected the age of the program (operational only three years) 
and the relatively small number of graduates (16%) would result in few career impacts. We expected the 
2002 study to demonstrate ^eater positive impacts on nurses’ careers. We were surprised to discover that 
among the two groups, the impact on careers was nearly the same across the two studies. If anything, the 
earlier group experienced slightly more career rewards than had the recent group, especially in the area of 
satisfaction with their nursing careers. 

Comparing CNP Impact on Nurses’ Careers 



2002 

215 

40% 

77% of respondents 



Promotion 


1999 Study 

10% 


2002 Study 

8% 


Increased responsibilities 


19% 


13% 


Change in employer 


16% 


14% 


Increased salary 


14% 


15% 


Change in area of practice 


18% 


17% 


Satisfaction with nursing career 


78% 


69% 



While CNP students were asked about future plans in both surveys, due to different wording of questions, 
only two areas can be compared. The 2002 group is somewhat more interested in supervisory or 
administrative positions, while the 1999 group is a little more interested in entering a masters program. 

Comparing Future Plans of CNP Students 





1999 


2002 


Pursue a supervisory or administrative position 


10% 


17% 


Pursue entry into a masters program 


52% 


49% 



Summary and Conclusions 

Both studies reveal a high level of satisfaction with the program; across all indicators, current and past 
students rate the CNP highly. The shift to an all Internet program has reduced barriers to access to the 
program, making a nursing baccalaureate degree a real possibility for working Wisconsin nurses. 
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Given the growth in and greater maturity of the program by 2002, we had expected to see mcreased career 
impacts attributed to participation in the program. Perhaps career impacts won’t be experienc^ for an 
even greater period of time, perhaps the indicators of career impacts we selected were insufficient, or 
perhaps respondents are reluctant to attribute these impacts to participation in the program. 
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Using the Northwestern University CoUaboratory 
With K-12 Students 

Bonnie Thurber 

Programs and Professional Development 
Northwestern University 

What Is the Northwestern University CoUaboratory Project? 

The CoUaboratory Project is a Northwestern University initiative developed to support the integration of 
Internet technologies into the K-12 curricula. The CoUaboratory was initiated with a four year, 1.8 million 
dollar grant from Ameritech in 1996. A team including a web designer, a programmer, and an educator 
was hired. The CoUaboratory Project team began by helping teachers learn how to use a browser and 
bring Internet resources to school for student research. Then, as a team, we supported teachers in putting 
student work on the web. Our K-12 teacher clients gave us feedback about what they felt would be useful 
applications of the Internet in the classroom. Internet-based software applications were created to answer 
the teachers' requests and we stepped out of teaching webpage creation and into dynamic ways of using 
the web to support teaching and learning. 

With teacher help, CoUaboratory Resources, including the following, were developed: 

Reading. Kids read books and write reviews. Peer recommendations are important. So we created "the 
Internet Book Club" as a place to select a book from a teacher's reading list and post a review. Other kids 
read the reviews to select the books to read. Kids can also contribute and read their own writing and 
poetry. 

Research. Kids enjoy getting sources from the Internet but lots of classroom time can be saved if the 
students have a list of pre-searched links from which to do their research. Teachers can create a "Cybrary" 
that is accessible from any Internet connection so students can do their focused research using teacher- 
recommended materials. 

Sharing. Webpages for sharing student work: Kids create beautiful art work, stories, and poetry, and they 
need a place to show it off on the web. When a classroom decides to put its work on the web, deciding 
what to post can be a manunoth job which includes finding an acceptable place to post the material, 
learning html, posting webpages, finding an appropriate audience to view and comment on the work and 
figuring out how to get those comments back to the students. MediaSpace was the CoUaboratory Project's 
answer to this challenge. In MediaSpace, students can contribute images, sounds, text and short videos to 
share with peers. Classrooms can build entire projects by copying and pasting writing work into a web 
form and clicking 3 buttons on an image/sound/movie contribution form. 

Data Collection. The Survey Studio was designed to enable teachers to design and administer surveys 
online. A web-accessible database stores results and dynamically generates webpages that display the 
results. Data can be analyzed in a variety of statistical formats. 

Discussion and Chat. WebBoard with WebMaster Conferencing was the first discussion forum used for 
asynchronous discussions. It also had an integrated chat function for real time interactions. 

Teachers used these tools in their classrooms as ways to conununicate what their students were doing in a 
large variety of projects. Some teacher comments include: 
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❖ “My students go to the computer lab and begin working on their research right away. They are 
interested in reading the web pages and they come back to class with notes that they can use in 
their papers.” 

❖ “Before the Cybrary, students did lots of searching and at best, came back with printed 
webpages that they needed to use for research.” 

❖ “They can look at other kids’ projects and get ideas of how theirs should work and look.” 

❖ “The data is great to have for teaching math and also for me to get an idea of what they know 
before we start an activity or unit.” 

Teachers came to workshops to watch us mentoring and modeling collaboration. Converting a unit into a 
web page became less im]x>rtant than the content and the students’ learning. The reason for posting 
curricula on the web is to provide an opportunity for other educators to see it and to use it. The reason for 
putting student work on the web is so peers can learn from each other. 

In 2000, we created and opened the Northwestern University Collaboratory, a laboratory for collaboration 
on the internet in which teachers create and manage engaging projects for their students and in which 
students contribute and share work. The webpages are dynamically generated. The focus is the 
curriculum, not webpage design. Standards were written to guide the curricula and the use of technology 
to enhance them. 

Our teacher clirats commented about the time it takes to get to after-school workshops, their tiredness 
after school, and their inability to always do their best work at the end of the day. Th^ do not like to miss 
school for workshops either. In answer to their concerns, we created complete documentation with 
illustrations and examples so they could work from home. Then, we created an eCourse which replaced 
our 15 horn workshop. The eCourse was designed to meet the needs and timelines of the teachers. The 
eCourse is available for educators who want to use the Collaboratory to develop technology-enriched, 
project-based learning activities that meet Illinois State Learning Standards and Goals for their students. 
Librarians, media specialists and technology facilitators who are collaborating with classroom teachers 
are encouraged to participate. 

Teachers said about the eCourse: 

“I enjoyed the class because I could do it on my own time. I could work late at night or early in 
the morning. When I finished the comse I had 15 hours credit and I had a project I could use in 
my classroom!” 

What has happened to the level of projects in the Collaboratory? Th^ were stronger, much better written. 
And, they are used in the classroom! 

Many teachers do enjoy creating their own units and curriculum. With the eCourse, they can do this on 
their own time at school or home. They can create the projects and activities with partners or by 
themselves. Th^ are working when th^ have their "stuff" available. 

The Next Steps 

In 2001, we started Collaboratory Sponsored Projects. Collaboratory Sponsored Projects are standards- 
based projects and activities that can be used in the classroom. Sponsored Projects may involve “experts” 
in a particular curricular or content area. Th^ can be developed by Collaboratory teachers, associates. 
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volunteers, or experts and may include such areas as seasonal themes, historical events, science, and 
community research. A Sp<msored Project can easily be “shared” by teachers and their students without 
having to create their own Collaboratory project. Collaboratory Sponsored Projects are “supported” by 
Collaboratory staff, associates, and/or volunteers. A sponsored project involving an expert was Kid- 
Made Toys From Around the World. Teachers who participated with their students say: 

❖ The teachers and students enjoyed this project and the toys they created. 

❖ An atmosphere of technical skills and collaboration was developed. 

❖ Primary teachers are interested in joining this project when it starts up again in March. 

❖ Several teachers have expressed an interest in creating their own projects within the 
Collaboratory. 

Another sponsored project created by teachers was Steps to Success: Creating Science Fair Projects. It 
impacted the classroom: 

❖ The Cybrary gave students a wide variety of science fair topics to choose from and resources 
for research. 

❖ MediaSpace contained examples which supported student topic decisions. 

♦♦♦ MediaSpace examples gave parents an overall picture of what students were expected to do 
in a science fair. 

❖ The SurvQ^ Studio survey gave students and teachers an overview of participant knowledge 
and experience. 

The Future Collaboratory 



K-20 educators create projects, activities, and resources they can use in their classroom with their 
students. Th^^ participate in projects written by other teachers. Th^^ form collaborations. Teachers, 
students, and experts mentor each other. 

In 2002, the Collaboratory Project received a grant from the National Educational Computing 
Association, NECA, to create a NEXUS for art, music, and writing in the Collaboratory. NEXUS meMs a 
synergy, a coming together, a connection, link, center, or focus. The Collaboratory Nexus application is 
being designed to encourage and support communication, mentoring, and creative collaborations through 
sophisticated Internet-based media and communication tools. It allows users to contribute, share, discuss, 
discover, and exchange digital information using video conferencing, chat, forums, and messaging and 
document posting and viewing. 
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Creating Virtual Conununities to Support Online Instructors 

Catherine L. Tompkins 
Academic Director 
The Art Institute Online 

Bradley S. Tompkins 
Program Director - Game Art & Design 
The Art Institute Online 

Wayne Batchelder 

Instructor - Online Task Force Chair 
The Art Institute of Dallas 

The proliferation of distance learning in higher education creates the need for new communication venues 
to support online faculty. Although, many published research texts provide adequate suggestions and 
recommendations promoting good techniques for online moderators, the research does not adequately 
address the unique and temporally sensitive needs of faculty members. The purpose of this workshop is 
to explore the aspects involved in the Art Institute Online’s effort in building a faculty lounge to support 
the social, developmental, and communication needs of a geographically separated university. The 
seminar examines the steps required to identify and establish goals; the logistics of building a faculty 
support area; and ideas to ensure faculty participation once the lounge is operational. 

Introduction 

Instructors who teach online courses are often isolated geographically fi'om one another and fi'om their 
hosting institution. The term “university” implies a community of learners and facilitators who 
contribute to the body of knowledge. On-ground institutions enjoy the ubiquitous exchange of ideas in 
commonplace venues such hall conversations, faculty lounge coffee discussions, lunch meetings and 
paper memos delivered in attendance folders or posted on bulletin boards. It is a challenge for online 
institutions to provide meaningful communication to serve the same purpose. 

How does an online school bring geographically dispersed faculty members together to inform, elicit and 
inspire educators to improve performance? Many online education resources focus on the student 
experience and curriculum materials for delivery of the course. When faculty accept the role of online 
facilitator, a host of new challenges ensue, including a change in instructional style, increased preparation 
time, and reduced interpersonal contact with students (Milheim, 2001). Failure to provide the basics in 
faculty interaction and support results in a host of undesirable consequences, including: social isolation; 
missing the big picture about how the university is structured; overlooked policies, deadlines, calendars 
and training opportunities; and the omission of shared knowledge and experience among faculty about 
their field and teaching in the online classroom. 

The kinds of experience faculty members can share with one another include online teaching tips, 
classroom experiences, and anecdotal lessons learned about unique encounters in class. In addition, 
participants experience consequential value in social contact and peer relationships through online work. 
The institution that recognizes these needs and provides adequate resources to meet them stands to 
dramatically improve faculty life and the quality of classroom instruction. 

The Art Institute Online (AIO), a virtual university affiliated with the Art Institute of Pittsburgh, offers 
degree programs in Graphic Design and Multimedia/Web along with a host of general studies courses to 
support on-ground Art Institute schools across the country. Students complete each course entirely online 
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through asynchronous web communication with faculty and fellow students dispersed throughout the US. 
Currently, over 100 adjunct faculty members serve as facilitators for AIO. In each 6-week session, 
students participate in class discussion a minimum of 5 out of 7 days. Faculty members are expected to 
check course discussions and answer questions within 24 hours. 

In addition to philosophical support by the institution, developers will require a coherent approach to 
develop an int^active web-based faculty support resource. Those steps entail how to identify and 
establish the goals and benefits of a properly implemented faculty lounge; the logistics of building the 
virtual space, including technology, personnel, and information resources; and ways to encourage 
participation and utilization of the tools provided. 

Current research in online faculty resources tends to focus on suggestions and recommendations that 
faculty members receive training and orientation about distance education, instructional technology and 
online facilitation. Short (2000) and Howell (2001) describe a framework for training and development 
which promotes learning for students with different learning styles. Finding out what kind of learner the 
students are can help to provide direction as to how online coursework should proceed during the term. 
Williams (200 1 ) and Howell (200 1 ) describe the importance of online discussion as a powerful tool for 
development, and recommend evaluating the difference in discussion-based learning against traditional 
lecture-based methods. The need for online faculty support materials and services is well-established by 
Howell (2001) who mentions studies that suggest the more time people spend online, the greater 
likelihood they will develop symptoms of loneliness and depression. Cagle (200 1 ) also mention the need 
for a dose of education theory in addition to technology application training when conducting faculty 
development activities. Lan found that the more administration provided technology and instructional 
support to faculty members, the greater their likelihood to become motivated and develop positive 
attitudes toward the learning environment (2001). Lee (2001) also showed that faculty motivation, 
commitment and satisfaction were greater when facilitators received instructional support for their 
distance education courses. Finally, Yucht (2001) describe the results of an institution’s successful efforts 
to build an online teacher lounge, where teachers can discuss personal and professional topics that shape 
the life of the classroom instructor. These studies support the notion that online faculty resources are 
warranted, encouraged, and supported, but do not provide ample evidence of successful implementation. 

Establishing the Goals 

The rationale for developing and implementing a faculty lounge should stem from a set of problems or 
needs the organization identifies. These problems can begin with general descriptions, such as “We need 
to increase communication between online faculty members, because they are isolated from each other, 
and from us.” A more specific set of problems help refine the required inclusions in the toolset, such as, 
“We cannot easily disseminate reference information such as job descriptions, referral forms, and contact 
information” or “The faculty need to follow a process for pre-reviewing their courses and letting us know 
any content or technical difficulties prior to the first day of class.” The kinds of problems that the faculty 
lounge addresses do not all need to be extant. The solution may proactively deal with issues before they 
arise. 

For AIO, the reasons to initiate a project to support online faculty began early in the program 
development. After starting up in 1 999, the school experienced rapid growth, both in student numbers, 
faculty members to teach them, and support personnel to develop, write and deliver curriculum. As with 
any (fynamic organization, the need to communicate to all parties is highly important, but traditional 
means were not available. Moreover, AIO’s operational model represents a major change from the 
syst^ to which most faculty are accustomed. For instance, some students attend AIO courses 
exclusively, gaining their degree entirely online. Many other students are “hybrid” students, attending 
on-ground courses at one of 1 1 schools currently participating in the project. 
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Finally, to manage, administer and direct students at a distance requires ongoing training and support. All 
AIO instructors undergo an initial 6-week training course using the same platform students encounter. 

Once instructors gain the initial orientation and begin teaching in the online environment, it is important 
to continue to offer additional training in advanced facilitator skills. 

Before undertaking a project that imparts widespread change for communication across the university, it 
is important to sit down with stakeholders and identify the goals. With targets identified, the project can 
move forward with the understanding that development is a process. As a living resource, it can be 
modified, extended, and improved. Although specific content topics will vary from one organization to 
the next, the general goals will likely include similar goals as those discussed in the next section. 

Goals of the AIO Faculty Lounge 

The Art Institute Online’s (AIO) education team members initially identified the following objectives that 
the online faculty lounge should address: 

❖ Increase communication with and among online faculty members. 

❖ Build an online learning community. 

❖ Provide a centralized location for reference information (job descriptions, contact information, 
policies, procedures, etc.) for online faculty members. 

❖ Provide basic online instructional support and development for faculty members. 

These goals were established with the understanding that as the faculty lounge grows, additional benefits 
and objectives will arise. A key design criterion is to create an architecture that is flexible enough to 
support future modifications. For this reason, the faculty lounge is a dynamic endeavor with no fixed 
end-result. Instead, the faculty lounge can grow to incorporate the needs of users and the hosting 
organization. Like some online courses, the faculty lounge should take on a life of its own, providing 
users with a rich, intCTactive environment. 

Identify the Potential Benefits 

Answer the question, “And what would that do for us (the feculty, the students, the courses, the 
administration, etc.)?” How will we know when we have been successful? Following are a list of ways to 
measure whether the faculty lounge project has achieved its stated outcomes: 

❖ Faculty will report that they feel a greater sense of community; 

* 1 * Faculty members find answers in the lounge spending less time searching for information; 

❖ Administrators receive fewer phone calls and emails asking for repetitive information; 

❖ Facilitators engage in faculty-driven professional development by sharing practical ideas for 
classroom managonent and problem-solving; 

❖ Facqlty members are able to cite example of techniques or approaches they have learned in the 
faculty lounge, techniques which improve their experience and the experience of the students; 

❖ The quality of education improves as measured by student participation, discussions, and 
student surveys; 

❖ Administrators communicate with the faculty in timely and effective manner. 

These outcomes help build the vision for what the online faculty lounge can be and can accomplish. This 
knowledge inspires the passion and provides motivation for the starting, continuing, and improving the 
resource. 
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Boilding the Faculty Lounge 



The practical steps involved in creating the online faculty lounge will vary depending on the technology 
resources available. The university may have extensive forums and discussion areas developed in-house, 
or they may rely on third-party products such as Blackboard or WebCT to create a suitable architecture 
for online lounge hosting within the delivery environment. These tools may significantly ditfer from 
those the students experience in the online delivery platform. 

The first step is to identify the technology available on the hosting system. Most of the goals stated for 
the AIO faculty lounge can be addressed with web pages and a threaded discussion forum. AIO used the 
Intraleam platform, hosted by Embanet to deliver both online classes and host the faculty lounge. In this 
way, faculty members experience the delivery mechanism from the role of student and participant, rather 
than facilitator only. Moreover, the interface is one to which participants are accustomed, and does not 
require conversion from or to another system. The link to the faculty lounge can be easily placed in the 
same area the faculty uses to access their courses. If additional technologies are needed to meet the 
objectives, such as chat-room, white-board discussion, voice and/or video, those resources should be 
identified and mobilized from the start of the project. The next step in this project is to ask the following 
questions: who will conduct ongoing coordination for site content and faculty lounge resources, and who 
holds the initial static information to be included on the site? The role of the coordinator is important, and 
the responsibilities may not be readily appar^t. 

The Coordinator 

Without a coordinator to drive participation and content, a virtual space will sit virtually unused. 
Regardless of the quality of the content, worthy intentions and adequate resources available, someone 
must be responsible to build, design, and post new content. Moreover, the coordinator must check in and 
administer the forum like a classroom, by welcoming newcomers, asking for meaningful input, and 
challenging the participants to dig into their areas of expertise to benefit others. The coordinator, should 
help create regular interaction opportunities for faculty members in order to facilitate sharing of ideas and 
building a cultiu*e of online participation. 



Conclusion 

Creating any substantive project requires setting and achieving a host of smaller goals that support a main 
purpose. Building an online faculty lounge is ultimately about helping instructors to improve 
instructional and learning performance in the classroom. Ongoing development and support will continue 
for many months. Underlying the purpose of the faculty lounge is the implied social contract that 
stimulates participants to improve and grow as a result of peer accountability. The commitment to 
improve is a joint responsibility, shared by administrators and faculty members alike. The faculty lounge 
is a support system and set of practices to help establish and achieve a more fully realized sense of the 
term “University”. 
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Background Information 

Since their introduction in the 1990s, distance-learning courses have grown dramatically in the number 
and variety of courses being offered. Today, both graduate and undergraduate courses are taught via 
distance learning (James, 1997). Many of these courses use the Internet as the primaiy means of 
communication between faculty and students, as well as among students, while others rely on a blended 
mixture of traditional classroom learning and distance learning methodologies. Phillips (1998) stated that 
“on-line universities” provide students with the flexibility to take classes at their own pace. A unique 
feature of much on-line learning is “asynchronous collaboration.” This allows students and faculty to 
interact with each other and the course material at a time and a place, which is convenient to them. This 
is in strong contrast to the synchronous environment that is prevalent in traditional classroom settings. 

Despite the many potential advantages of Internet-based courses, there has been a lack of systematic 
research on the relative effectiveness of such courses (Moesel, Dass, Werner, & Pouts, 1998). Kroder, 
Suess, & Sachs (1998) described issues they faced in launching a web-based graduate course in a 
collaborative effort between the University of Texas-Irving and Pace University in New York. For 
example, both students and faculty felt the web-based course required substantially more time than a 
traditional course in a classroom. Nevertheless, 80 percent of the students said they would take another 
on-line course in the future. James (1997) described a survey of MBA graduates of the University of 
Wisconsin- Whitewater that analyzed their reactions to a planned emphasis using on-line course offerings. 

In the search of the literature to date on distance learning, only limited research was found (University of 
Idaho, 1995). For example. Sherry, Fulford, and Zhang (1998) devised a measure of satisfaction with 
instruction for distance learners. Arbaugh (2000) found that perceived usefulness of the learning software 
as well as the perceived flexibility of taking courses on-line were both related to student satisfaction with 
their on-line courses. 

A review of the American Society for Training and Development’s (ASTD) Web site (www.astd.org) 
finds a great deal of information on e-leaming, including a “Daily News” section, which features articles 
on e-leaming appearing in each days press. A sampling of two days (June 4 & 5) yielded an average of 
17 articles per day on the subject. Many of these are related to vendor issues, several offer a very 
“observational” approach to how to and how not to do e-leaming, but none of them offer any hard data on 
e-leaming programs. ASTD also offers an electronic “webzine,” an Internet magazine, called Learning 
Circuits which deals exclusively with the topic of electronic learning (www.leamingcircuits.org). A 
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search of this web site offers a lot of hands-on tips for e- learning, but very little actual research on how to 
make it more effective. 



The University of Wisconsin- Whitewater began planning an On-Line MBA program in the fall of 1996. 
After researching several long-distance delivery systans, LeamingSpace from the Lotus Corporation was 
selected as the primary instructional tool. In LeamingSpace, students can access courses using an Internet 
browser such as Netscape or Windows Explorer or they can access the course using a Lotus client. The 
Lotus client is loaded directly on the computer hard drive for faster access to the program. In addition, 
students purchase CD-Roms, which contain instructor developed support material and course information. 



The purpose our project is to explore On-Line learning in more depth than that of previous work. Due to 
the evolving nature of the On-line program itself, our research has also changed over time. Our primary 
research tool is a survey administered Administering surveys to students who take classes in the On-Line 
MBA program does the research. In the most recent phase we also give a survey to students in a 
traditional MBA program. This session traces the development and evaluation of our research. 



The sample population for this study consisted of students enrolled in courses for the On-Line MBA 
program at Ae University of Wisconsin- Whitewater during the fall 1998, spring 1999, and fall 1999 
semesters. The survey was given to 206 students in 10 different MBA classes, all of who met 
electronically. As independent variables, six issues were tested to determine their impact on student 
perceptions of the on-line course experience. These are: 

1 . Perceptions that training was sufficient 

2. Perceived problms accessing the course 

3. Accessing the course via the notes client server vs. an Internet browser 

4. Accessing the course from home, school, or work 

5. Part-time vs. full time employment status 

6. Attending school part-time vs. full-time 

For the dependent variables, a factor analysis of the 3 1 remaining questions on the survey revealed five 
major issues. The results of this factor analysis identified the following five factors: 

❖ Level of communication with the instructor 

❖ Level of technical expertise and assistance needed 

❖ Interactions with other students 
»J» Effectiveness of delivery method 

»> Likelihood you would recommend this course to others 

1. Does the amount of technical training received impact student perceptions? 

Students who felt they received adequate technological training had more positive feelings concerning the 
five issues studied. Statistically significant differences were found concerning the level of 
communication with the instructor and the perceived effectiveness of the delivery method, such that those 
who felt that they had been adequately trained were more positive about both issues. Interestingly, the 
relationship between training and interaction with other students approached statistical significance, with 
students who felt more positively about the training were also more likely to be positive about interaction 
with other students. It should be noted that students in general had overall positive perceptions 
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concerning each category, i.e., the first four dependent variables were measured using a seven-point scale, 
and all means were above 4.0. The last variable, “recommend course to others,” was measured using a 
yes/no item, thus ranging from 1 to 2. This may explain why there was little difference between the 
groups concerning whether or not the students would recommend the course to others. 

2. Do access problems impact students’ views of on-line courses? 

Experience has proven that ease of student access is a critical factor for on-line courses. Students (and 
faculty) quickly get fhistrated with new technology that does not work properly or is not user-friendly. 
Moesel et al. (1998) suggested students who had greater problems accessing the course were less positive 
about on-line courses. The student’s ability to access the course was significantly related to the perceived 
level of technical expertise needed for the course. Additionally, the relationship between access problems 
and two other variables approaches statistical significance, namely, level of communication with the 
instructor and recommending the course to others were both more positive when access problems were 
less of a problem. 

3. Does the use of the Notes client versus the Internet to access their course impact student 
perceptions? 

Students were able to access courses in two ways, via Lotus Notes off their hard drive or via the Internet. 
Does the method of access make a difference? None of the dependent variables were significantly related 
to whether students used the Notes client versus the Internet to access their course. This is surprising, 
given the delays that often occur when students must use the Internet to access their courses. However, 
for this sample at least, means of access was not strongly related to any of the outcome measures. 

4. Does accessing a course at home, work, or school impact student perceptions? 

How students access the course was significantly related to their willingness to recommend the course to 
others, with students who could access the course from work more likely to recommend the course to 
others. Additionally, students who could access their courses from work were more positive about the 
level of technical expertise needed, although this relationship only approached statistical significance. 

One of the many benefits of on-line courses is the time flexibility. A student’s ability to access the course 
prior to work, during a mid-day break, or after work appear to find this flexibility to be a great benefit. 

5. Are student perceptions impacted by employment status? 

In general, student’s employed fiill-time had more positive attitudes towards on-line courses. Full-time 
employment status significantly impacted whether or not students would recommend the course to others. 
Further, the relationship between fUll-time work status and the perceived effectiveness of the delivery 
method approached significance. This could also relate to the issue of flexibility. Full-time workers 
taking a graduate program at night value the ability to fit the course of study into their schedule, and thus 
develop more positive perceptions of a program that allows for this flexibility. Full-time workers also 
don’t have the time to ‘learn how to do it’, so sufficient training and assistance is more critical to them. 

6. Are student perceptions impacted by full- versus part-time student status? 

As expected, part-time students generally perceived on-line courses more positively than full-time 
students do. Part-time students were significantly more positive about the on-line approach as an 
effective delivery method, and wa-e more likely to recommend the course to others. Although not 
statistically significant, part-time students held more positive feelings concerning level of communication 
with the instructor and level of technical expertise required. These findings support current literature that 
suggests that one of the major benefits of on-line courses is that active people can benefit from the time 
flexibility of on-line courses. On-line courses allow students to do course assignments when work, 
family, and other responsibilities are not as pressing (Phillips, 1 998). On the other hand, full-time 
students did hold significantly more positive perceptions concerning interactions with other students. 
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This is likely because full-time students were on campus for other classes and therefore were able to meet 
in person with each oth^ for group projects and other assignments. 

Study 2 

A second study (still in progress) was begun in fall, 200 1 . The survey was given to 320 students enrolled 
in 14 On-Line MBA courses and to 246 students enrolled in 16 traditional (classroom) MBA courses. 

This study used the following factors: 

❖ Instructor interaction 

❖ Course structure/eflfectiveness 

❖ Course content 

❖ Interaction with other students 

»I» Overall satisfaction with the course 

While this study is classified as a “work in progress” it is interesting to note that on all five factors noted 
above, the on-line courses were rated higher than the traditional courses. While some of these differences 
were not great, for three of the five factors the differences were statistically significant (course 
structure/effectiveness, course content, interaction with other students). 

Conclusions 

While these research projects are on going they do offer some tentative conclusions about on-line 
learning. Our most recent findings - that on-line learners have as high or higher satisfaction with the 
factors of the course are, of course, based on a relatively small sample, but do suggest that on line 
learning can be a very effective teaching tool. Our findings also support the idea that flexibility is a key 
issue in student preference for on-line learning. We are also finding that (like most areas) there is a strong 
learning curve, both for students and for faculty. Initially, concerns with technology dominated a lot of 
thinking, but through effective training, support and interaction these issues appear to diminish 
significantly. 
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Introduction 

A prototype system of web sites containing several different subject disciplines, executable 
instruction sets and reusable concept object modules has now been extensively utilized and 
evaluated. A diverse student population, liberal arts majors, non- traditional students, and 35% 
Hispanic mix in contemporary math, calculus III students, and linear algebra students was the 
composition of our test sample. The performance of the system has been flawless and the student 
response (interviews and written evaluations) has been excellent. Executable mathematical 
functions, dynamic graphics, and concept-oriented animations have generated positive feedback. 
We are now working to bring the system to a fuller expression of its potential by the addition of 
more disciplines, reusable concept object modules, and executable instructions. It is time to 
broaden the base of utilization by providing workshops for K-12 schools in our area, local 
community colleges, and the other branches of the University of Texas System. The Assistant 
Vice Chancellor of Academic Affairs who is also the Director of the UTTeleCampus has given us 
a letter of support for this project. Presentations have been given at several conferences in which 
this system was demonstrated and abstracts have been accepted for additional presentations this 
year. Thirty-seven of thirty-eight liberal arts majors said the sites were helpful and that they 
would use them if they had access from their home computers 

Description of the Web Sites 

The web site has three major components. Subject disciplines, listed in a left navigation frame. 
Each major subject discipline has a right frame page with a link to a facilitator sites composed of 
10 to 12 executable instructions, which are conunonly used in the study of the subject discipline. 
These are not applets. Also found on this page are links to reusable concept object modules that 
are similar to chapters in a textbook dealing with some concept found in the study of the subject 
discipline but these reusable concept object modules are highly interactive unlike the traditional 
text. The prototype is a suite of web materials forming a web-accessible library of 
mathematically oriented content object modules. Authenticated access to the site will be made 
available to interested colleagues at other institutions. Once inside the site, the user sees a 
navigation frame on the left and a larger frame to the right. In the left frame or navigational 
frame the subject disciplines, which are all links, and an inventory of the supporting executable 
instructions in the facilitator sites are found. Clicking on the subject discipline link brings up the 
main page for that subject discipline in the right frame. A click on the words Linear Algebra in 
the navigational left frame brings up the main page for Linear Algebra in the right frame. This 
right frame contains three major items, a link to the facilitator site for linear algebra, a detailed 
graphic showing how a matrix must be set up to input it to the executable instructions, and a 
group of links to the available concept object modules on topics from linear algebra. As the user 
accesses the main subject page or moves from there to facilitator sites or to the associated object 
modules, the navigation frame stays visible. This is an innovative feature of the system that 
allows a user working on the linear algebra facilitator site main page site, or any of the linear 
algebra object module sites to branch immediately to any of the other subject areas appearing in 
the navigational left frame. An integral calculus student deq) into an object module dealing with 
the concept of definite integrals as probabilities can, with the click of a mouse, reach the statistics 
site if desired. The ability to easily transfer from one subject discipline to another is one of the 
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major innovative features that make this suite of web sites an integrated multidisciplinary system. 
Each subject area has a similar arrangement. This organization makes the whole system user 
friendly, versatile, and easily expandable in three ways. It is a trivial process to extend the 
number of subject disciplines. Also the executable instruction sets can be easily expanded or 
created. As the object modules are completed it is a simple task to add them to the system by 
inserting the links on the main page for the discipline under study. 

Current Inventory and Utilization of the Web Site 

The web site presently has the following course disciplines available: linear algebra, calculus, 
statistics, college algebra, and some utility sites composed of special functions such as a set of 
several plotting routines and solving functions, algebraic and differential equations. All together 
we have about 50 executable instructions available that our students can access. These 
executable facilitator sites are part of the higjily interactive parts of the reusable object modules. 
The students working in a linear algebra facilitator site or object module can with ttie click of a 
mouse transfer to the calculus facilitator site to do an integration, the plotting facilitator site to do 
a plot or use one of the linear algebra functions such as row reduction or to another object module 
in another subject discipline. These sites are called facilitator sites because they facilitate the 
checking of ideas and the correctness of homework attempts. Each facilitator site has a complete 
set of instructions explaining how the input must be typed and arranged in the input window of an 
executable instruction, a drop down menu with the names of the executable instructions, and then 
an evaluate button. A user types the expression to be operated on into an input window, clicks on 
the drop down menu, selects the executable instruction desired, and then clicks on the evaluate 
button. The Mathematica kernel in the server does the operation and the results are instantly sent 
to the client’s computer for observation. 

Development of the Executable Instructions in the Facilitator Sites 

These highly interactive facilitator sites used by the object modules are made possible by using a 
new version of Mathematica released this year by Wolfram research, webMathematica. The 
kernel, which is the computational engine, is stored and executes in the server. A Servelet 
engine, Apache Tomcat 4, and J/Link, which shows Java how to understand the Mathematica 
language and Mathematica how to understand Java, all work together in a Java run time 
environment so that a Math Server Page can be used to initiate a Mathematica session in the 
server and return results to the client. The Math Server Page (MSP) is an html page containing 
some special tags <% — %> enclosing some code called mathlets and can be identified by its 
“.msp” extension. The mathlet and the extension activate the Mathematica kernel through action 
of J/Link and the Servelet engine. The computation is then done in the server and not the client 
computer. The result is then sent to the client’s browser where the result is displayed. 

Development of the Reusable Concept Object Modules Work 

In addition to the facilitator sites we have designed a variety of reusable concept object modules. 
These concept object modules are initially produced as Mathematica notebooks. Then, most of 
the Mathematica instructions are stripped out and the remaining part is partitioned into small 
segments. These segments are then stored with the appropriate extensions so that IBM’s 
techexplorer will interact with them when they are loaded into the browser. Before finally 
uploading to the server, linkage code is added to each partition so that the user can communicate 
directly with the facilitator sites if desired from each site page of each object module. These 
concept object modules are accessed from the main pages for the different disciplines as was true 
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for the facilitator sites. For instance, when on the linear algebra main page, the student can click 
on the Matrk Operation link and transfer to a set of web pages that provide detailed illustrations 
of different matrix operations. These explanatory pages have text, graphics, animations, and in- 
line executable instructions that allow on the spot computations. As another example, the 
calculus main page has a link to a concept object module that discusses line integrals. The 38 
pages of this module include explanatory text, examples, graphics, animations, dynamic 
animations, practice exercises with executable instructions, and the solutions to the exercises. 

The object modules use IBM’s Techexplorer to interface with the Mathematica kernel. This 
inexpensive software makes possible inline executable instructions. Techexplorer also 
understands MathML and therefore allows the transfer of mathematical symbols to the user’s 
screen. So far we have developed sbr of these concept object modules. 

Alternatively, and an approach now under investigation, is to use Word 2000 with MathType 5.0 
to develop the object modules in html. Then we add the needed links to special executable 
instructions and the facilitator sites. This approach holds promise and does away with the need 
for a third party program, techexplorer. 



Evaluation Results 

The facilitator and object module sites have worked well and have been given positive 
evaluations by our students in linear algebra and calculus 3. Out of 45 evaluations in linear 
algebra, there were some suggestions as to how we could improve our sites but there was no 
negative feedback. I have included several specific student comments that represent a cross 
section of the over all student response to our web site. 

Linear Algebra Evaluation 

❖ “I love the website. I’ve used it for homework & study in this class & in Programmer 
Algorithms (Note: this is a senior level computer science course). It’s faster and more 
user fi’iendly than Mathematica and you can access it via the Internet. The visual 
graphics make the graphs come alive and show us what is happening in reality. Being 
able to move them in 3D fashion is a great plus.” 

•> “I thought that the website was of great help. I was able to log on quickly and without 
hassle. I like that I was able to access it from home, anytime I needed to. The website 
was simple to use. I did not have to spend hours frying to figure it out, like some 
websites out there. The graphics on the computer helped me to see what was 
happening. I am a visual learner. Most professors do not spend the extra time 
developing projects like these. I found it very helpful.” 

❖ “The math web site helped me out numerous times. It is so refreshing to be able to 
check your answers. I could get a better understanding of how to do assignments thru 
this web site. I would highly recommend this web site to anyone who is in any math 
class. The site boosted my average in the class by at least one letter grade.” 

The web site was veiy useful in helping me to understand this course. It was very easy 
to use and I feel contributed to my knowledge in this subject. The visual graphics you 
used during class really helped me “see” what was going on rather than viewing a 2- 
dimensional picture in the text book.” 

»> “The website is Awesome! I was going to ask if we came back next semester an asked 
for the password if we could continue to use it for reference! I would love to see other 
math areas integrate COSC techniques into their classes. The graphics brought the 
concepts to life! It was so great to be able to see the orthonormal vectors in 3-space. It 
helped my understanding tremendously!”Great website, I don’t know of any other 
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helped my understanding tremendously!”Great website, I don’t know of any other 
website that offers so many different ^ings. It’s easy to use, very user-friendly and 
everything that you need is there. — Keep it up.” 

Conclusion 

When a student reads the explanation of a mathematical concept in an object module, he or she 
might be requested to plot, differentiate, integrate, or take a limit of a function, or perform 
regression analysis or any number of other activities. All of these operations, plus many others, 
are now interactively possible using an adaptation of the new technology described in this paper. 
This freely open interactivity pomits student exploration and thereby opens a window that 
infuses education with the excitement of discovery. It has been proven repeatedly that hi^ 
interactive learning modules, especially those with high visual content result in significantly 
improved learning gains [1, 2, 3, 4, S, 6, 7, 8]. Stud^ts can investigate ideas of their own to 
determine how changes may affect the outcome of a calculation or of a graphical picture that they 
might never attonpt by hand. The interactive functions can be defined for a concept in any 
course in science, mathematics, engine^ing, or technology. Th^ the text, graphics, etc. are 
developed to describe the concept for the appropriate subject discipline. In the concept object 
modules in-line executable instruction are integrated into the text, providing a highly interactive 
teaching tool. If students work in groups using the computer, this approach will help foster more 
collaboration. 
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The Evolving International Role of a Research University: 

An MIT Perspective 

Janet Wasserstein 

Communications and Development Officer 
Center for Advanced Educational Services 
Massachusetts Institute of Technology 

The role of a research university today, compared to fifty years ago, now needs to be aware of the global 
picture and interconnectedness. If tertiary education leads to greater understanding internationally and 
empowers students, what role should a research university play? At MIT, several programs are currently 
underway that address this question. 

MIT offers some distance learning programs but does not offer degrees solely through distance learning.' 
There are other universities that concentrate on that area. MIT is interested in researching new 
educational technology such as interactive online tutors (e.g. PIVoT"), and initiatives such as the Open 
Knowledge Initiative and Stellar’". MIT is also looking at new pedagogies that are goal-oriented, non- 
linear and experimental and new ways of collaborative learning such as the Technology Enabled Active 
Learning project (TEAL’'). 

MIT employs many different models for distance learning or technology-enabled education as part of 
teaching tools that have global dimensions. MIT provides outstanding content by its faculty and 
innovative research in ways of teaching and learning. 

MIT Models of Global Learning Initiatives 



Publishing Model 

The latest project that has received much publicity is OpenCourseWare. This is a project to put MIT 
course materials up on the web free and open to Ae public. This is not distance learning, however, as no 
interaction occurs between teacher and learner, but it is an invaluable resource for students and teachers 
across the globe. “The idea behind MIT OpenCourseWare (MIT OCW) is to make MIT course materials 
that are used in the teaching of almost all undergraduate and graduate subjects available on the web, free 
of charge, to any user anywhere in the world. MIT OCW will radically alter technology-enhanced 
education at MIT, and will serve as a model for university dissemination of knowledge in the Internet age. 
Such a venture will continue the tradition at MIT and in American higher education of open dissemination 
of educational materials, philosophy, and modes of thought, and will help lead to fundamental changes in 
the way colleges and universities engage the web as a vehicle for education. MIT OpenCourseWare will 
provide the content of, but is not a substitute for, an MIT education. The most fundamental cornerstone 
of the learning process at MIT is the interaction between faculty and students in the classroom, and 
amongst students themselves on campus.”'’ 

MIT Learning Networks 

Learning networks are used in many different programs at MIT and are defined as spatially dispersed 
communities connected together by a variety of communication technologies where everyone is a learner 
and a teacher, and many opportunities exist for collaborative research'^. Examples of these partnerships 
include the Singapore MIT Alliance (SMA) and the Cambridge MIT Institute (CMI). 

Using asynchronous and synchronous technology as well as on-campus immersion, the Singapore MIT 
alliance is MIT’s largest distance learning project and is an innovative engineering education and research 
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collaboration between the National University of Singapore, Nanyang Technological University and MIT 
(http://web.mit.edu/sma). 

This model can best be described as follows: “At MIT, we are using the broader bandwidth Intemet2 to 
deliver five new professional masters degree programs to students on our Cambridge-based campus and 
to students 12 time zones away in Singapore. But beyond formalized distance learning, SMA involves 
collaborative faculty research, student exchanges and a wide variety of activities that result in a type of 
global campus spanning the Pacific”."' 

Current MIT Initiatives From CAES to Reach Global Audiences 

MIT is a community of over 10,000 students, faculty and staff. In 2001-2002, MIT’s international 
student population comprised 8% of undergraduates and 38% of graduate students from 109 different 
countries. MIT students need to be prepared to enter a global economy and MIT is connected to research 
institutions all over the world. The MIT International Science and Technology Initiatives (MISTI), for 
example, have cultural exchange and global programs that have reached thousands of students worldwide. 
This paper can only touch on a few of the current and planned initiatives from MIT and the focus of this 
paper will be on the initiatives of CAES - one of MIT’s main centers of technology enabled education 
and distance learning - that collaborates with other MIT schools and departments. 

Using some of the models described above, over the last year, CAES has increasingly focused on 
disseminating its knowledge globally particularly to developing countries through distance learning. 

The reasons for this outreach are several: the world is transitioning from an economy based on 
manufacturing and natural resources to one that is more focused on information and knowledge; 
developing countries have become very active in web based distance learning (e.g. Pakistan, Uzbekistan, 
Tunisia, Algeria, Jordan, South Afi’ica, Tanzania, Iran); and technology can increase access to larger 
audiences. 

The following three projects are the latest initiatives of CAES (http://www-caes.mit.edu/index.html) : 

MIT World 

One of the most successful projects of CAES, MIT World follows a similar model of OpenCourseWare to 
bring content to global audiences. Although there is no interactivity, intellectual content from MIT can 
now be seen from Mali to Massachusetts - or indeed by anyone with a computer, Internet access and 
RealVideo player. MIT World is a free, open, video streaming web site that currently provides on- 
demand video of significant public events at MIT (http://web.mit.edu/mitworld) . It is the venue for the 
MIT community, alumni, and the public at large to have access to video taped events at MIT. Content 
includes: 2001 Nobel Prize in Physics Lecture by Wolfgang Ketterle on "The Coldest Matter in the 
Universe" and Nobel Laureates David Baltimore, John Hume, Charles Townes, Franco Modigliani, Paul 
A. Samuelson, and Robert M. Solow. Lectures cover a broad range of subjects including the arts, 
economics, politics, computer science and management. 

Africa Virtual University 

More than 190 students from eight sub-Saharan African countries (Ethiopia, Ghana, Kenya, Mozambique, 
Rwanda, Tanzania, Uganda and Zimbabwe) participated in an MIT course without leaving their continent, 
through a partnership between CAES and the African Virtual University. Using asynchronous and 
synchronous methods, CAES offered a six-week curriculum loosely based on MIT’s Introduction to 
Computers and Engineering Problem Solving (1.00), which teaches the Java programming language. The 
resulting course, Java Revolution, was uniformly distributed regardless of equipment and bandwidth. 
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Thirteen learning centers participated, and we expect to enroll more African students in MIT courses in 
the future. Java Revolution featured videotaped lectures via satellite, a web site for course materials, e- 
mai! moderated by teaching assistants, and two live videoconferences. 

Two Interactive Web-Based Courses 

With financial support from Pfizer, CAES and the Harvard MIT Division of Health Sciences and 
Technology have created the first two in a series of free, interactive web-based courses for healthcare 
professionals: “Good Practices in Clinical Research” in Spanish and English primarily for physicians in 
Latin America; and Fungal Infections: Virtual Grand Rounds as a response to a disease that is endemic in 
many developing countries (http://figrandrounds.org/fi/index.htmn 

Learning International Network Consortium (LINC) 

The Director of CAES, Professor Richard Larson, is the Principal Investigator for a new project that we 
hope will receive funding this summer and will begin its first year of planning by September 2002: the 
Learning International Network Consortium (LINC). Using the train-the-trainer model, CAES plans to 
bring world-class educational opportunities through distance learning to those who have not had access to 
quality education. 

LINC is an MIT-managed initiative aimed at the creation of a global community of scholars and 
practitioners who seek to use modem computer and telecommunications technologies to increase the 
reach and impact of higher education especially in developing countries. LINC will be a voluntary e- 
leaming network and resource center connecting and building upon individual national and international 
efforts currently underway. Using e-leaming and related technologies, LENC will empower educators 
working in their own country to bring higher education to a wider population. Students of current ‘brick 
and mortar’ university programs will Join faculty members and othws in collaborative research and 
development to help create positive change. Educational content will focus primarily in the areas of 
science, engineering, management, humanities, arts and social sciences. 

The key purposes are to empower organizations within countries, especially developing countries, to 
design, create and operate top quality higher-education e-leaming systems; to share educational materials, 
exercises and courses for the mutual benefit of learners across all participating countries; and to become 
the top tier community of scholars and practitioners whose focus is technology-leveraged higher 
education internationally. 

The Consortium will be assembled from diverse efforts now operating within many developing countries, 
utilizing best pedagogical practices whenever possible. LINC will bring world-class educational 
opportunities to those who do not have access to quality education by partnering with and training 
teachers in developing countries, who will then work within their own country to increase outreach. At 
the same time, those in developed countries will have the opportunity to learn about different cultures 
through the educational networks established by LINC. Members of LENC will be colleges and 
universities, not-for-profit organizations, foundations, private corporations and selected governmental 
agencies. Expanding and sustaining e-leaming efforts throughout the developing world will be the 
primary focus of LINC. The reasons behind the initiative include: 

❖ Developing countries are likely to fall further behind the developed world in terms of economic 
growth and prosperity, without some feasible system for educating their youth. This in turn 
could lead to larger inequities in the distribution of the wealth of nations and potentially act as a 
destabilizing factor in world events. 
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❖ Establishing a two-way exchange of content between the developing regions and the developed 
world will enhance mutual understanding and provide a rich source for cultural studies. 

Some of the specific goals and services of LINC are as to: 

❖ Host an annual international conference on “e-Leaming in the Developing World” that would 
share “lessons learned”, present best practices in e-leaming and project plans for future years. 

❖ Design the management and membership protocols and processes of LINC, leading to a 
sustainable and readily growing organization. 

*t* Create and distribute a variety of educational content both intra-country and across national 
boundaries. 

❖ Using the train-the-trainer model, design systems and utilize technology to leverage scarce 
teaching resources within and between countries to reach a maximum number of learners. 

LINC would provide resources that would focus on training those who in turn would train 
others in the technology, pedagogy and business of e-leaming. 

*!* Create and maintain an easy-to-use web platform for communication among LINC participants 
and for assisting in the hosting of educational materials. 

❖ Host a complete searchable repository of international e-leaming projects and activities that are 
underway, so participants can identify potential partnerships and other synergies. 

*!* Establish a global network of learners to advance the sharing of skills and raise economic 
standard of living. 

*> Raise the educational level of population globally. 

❖ Undertake collaborative research and development, involving faculty and students at existing 
institutions of higher education, to further this effort. 

Conclusion 

CAES plans to leverage its educational technology projects and direct much of its attention globally to 
developing countries.™' Distance learning and technology-enabled education will allow increased access 
to tertiary education — a scarce resource in many countries. 
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‘ MIT oflFers: one degree (Systems Design Managemait Program) through distance learning that requires one 
semester on campus; oflF campus computer and engineering for credit graduate certificate courses through CAES’ 
Advanced Study Program; and short non credit courses through the CAES National Technological University 
partnership. 

“ For details about PIVOT see http://www-caes.mit.edu/research/pivot/index.html 
“ For details see http://web.mit.edu/amps/projects/index.html 
For details about TEAL see http://www-caes.mit.edu/research/teal/index.html 

OpenCourseWare Fact Sheet httD://web.mit.edu/ocw/ocwfactsheet.html . See also Report of the President 2000- 
2000 1 http://web.mit.edu/president/communications/rpt00-0 1 .html. 

Larson, R. C., ”MIT Learning Networks: An Example of Technology-Enabled Education.” September 1997. 
http://www-caes.mit.edu/people/larson/leamingnetworks.pdf 
Larson R.C. and Strehle G.P. (2002) Edu-Tech: What's a President to Do? Lawrence Erlbaum Associates 
A discussion group fi-om a virtual conference on The Future of Technology in Developing Countries connecting 
over 260 participants came up with the following findings: cultural and religious issues must be considered before 
any new Vocational strategy is implemented in developing countries; radio is a basic communication tool that 
serves education especially in areas where there is lack of appropriate bandwidth; and >^ile English is widely used, 
one must use sensitivity to language issues and that many have difficulty with English as well as accepting new 
educational technology tools (see www.col.org) 
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Experiential Learning Activities in Distance Education: 
Challenges and Examples 



Doreen Westera, MscN, MEd 
Associate Professor 
School of Nursing 
Memorial University 



One of the greatest challenges faced in developing and teaching a course for Distance Education Delivery 
is to actively engage students with the course material so that their learning is maximized. This paper 
explores the potential for experiential learning activities in distance education. 

Common themes in descriptions of experiential learning include the centrality of human experience as a 
valued source of learning, the potential for every experience to be a learning situation and the central role 
reflection plays in learning (Bumard, 1993, 1990, 1989; Kolb, 1984,; Pfeiffer & Goodstein, 1982). Three 
characteristics of experiential learning that are relevant to a framework for course development and 
delivery include; experiential learning as an attitude toward learning - questioning, problem posing, 
testing ....; experiential learning as a form of knowledge, as legitimate as propositional knowledge 
(textbook) and practical knowledge (knowing how); and experiential knowledge as learning through 
experience and learning from experience (Garratt, 1983; FEU, 1983; Keeton & Associates, 1976; Dewey, 
1938). 

Experiential learning cycles can be used as a basis for planning activities. For example, Bumard (1989) 
describes a cycle involving five stages: brief theory input, concrete experience, reflection, generation of 
new knowledge and application. An illustration from an assignment in a course developed and taught by 
the author shows how this cycle can be applied; students receive a brief explanation of spiritual 
assessment (Stage 1); they complete an audiotaped spiritual assessment with a ‘client’ (Stage 2); through 
a series of questions they are stimulated to reflect on this assessment - their thoughts, feelings, appraisal 
of the assessment questions (Stage 3); they discuss the new knowledge generated by this activity in a 
paper (Stage 5). The same cycle can be applied to the educator as she reads/listens to and reflects on the 
student’s presented work. 

In addition to assignments, experiential activities can involve the use of reflective questions and exercises, 
simulations/games, and videotaped scenarios. In all cases, if these are being evaluated for grading, the 
quality of the reflections should be the focus of grading, not the activities in and of themselves. 

Challenges in designing and delivering experientally oriented activities for distance education include; 
fitting such activities within the curriculum framework of the program/course; planning activities which 
can receive their maximum benefit from distance education technologies; self-reflecting on one’s own 
philosophy of teaching/leaming to ascertain if it is consistent with the philosophical underpinnings of 
experiential learning; and evaluating experiential learning activities in a relevant and meaningful manner. 

Experiential learning activities can greatly enhance learning for both student and educator. Such 
activities promote a partnership stance between the two. They increase student awareness, are 
real/whole/interactive, can be light-hearted and fun, and foster critical thinking. It is important to 
remember, however, that some students may find such activities to be uncomfortable or threatening and 
may prefer other means of learning. 

In being involved in experiential learning, we may confidently agree with Dodo in Alice in Wonderland. 
When Alice asked the animals and birds that had fallen with her into the pool of tears how to get dry, the 
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Dodo told her that a caucus race would be the best thing. “What is a caucus race?” asked Alice. “Why,” 
said the Dodo, “the very best way to explain it is to do it!” 

Reference List 

Bumard, P. (1993). Using experiential learning methods with larger groups of students: Network 
heading. Nurse Education Today 13, 60-65. 

Bumard, P. (1990). Learning from experience: nurse tutors ’ and student nurses ’perceptions of 
experiential learning. PhD thesis. Cardiff: University of Wales College of Medicine. 

Bumard, P. (1989). Teaching interpersonal skills: A handbook of experiential learning for health 
professionals. London: Chapman and Hall. 

Dewey, J. (1938). Experience and education. London: Collier Macmillan. 

FEU. (1983). Curriculum opportunity: A map of experiential learning in entry requirements to higher 
and further education award bearing courses. London: Further Education Unit. 

Garratt, R. (1983). The power of action learning. In M. Pedlar (Ed.), Action learning in practice. 
Aldershot: Gower. 

Keeton, M. & Associates. (1976). Experiential learning. San Francisco: Jossey Bass. 

Kolb, D. (1984). Experiential learning. Englewood Cliffs, NJ: Prentice Hall. 

Pfeiffer, J. W. & Goodstein, L. D. (1982). The 1982 annual for facilitators, trainers and consultants. 
San Diego, CA: University Associates. 

Biographical Sketch 

Doreen Western is an associate professor at th School of Nursing, Memorial University. She has been 
involved in nursing education for over twenty years, teaching in the areas of community health nursing, 
health promotion, counselling, and spiritual dimensions of nursing practice (the latter two courses by 
Distance Education mode). Professor Western has also developed four videos which are currently being 
distributed in Canada, United States, Australia and the United Kingdom: Spirituality; Nurses and Spiritual 
Care; Spiritual Assessment; Community Assessment. Additionally, she is a Certified Canadian 
Counsellor and has a small clinical practice at a community agency. 



Address: 


School of Nursing 




Memorial University 




St. John’s, NL AIB 3V6 


Email: 


dwestera@mun.ca 


Phone: 


709.777.7259 


Fax: 


709.777.7037 



438 ❖ Experiential Learning Activities 



Are You Practicing What Research 
Is Preaching for Distance Learning? 

Kay Wijekumar, Ph.D. Assistant Professor 
School of Information Sciences and Technology 
The Pennsylvania State University Beaver 

Introduction 



Web-based distance learning environments are growing at an exponential rate but research studies to 
support the developments are limited. For example, when we review the research on reading and 
comprehension there are volumes of research studies and many methods that have been tried and 
discarded or adopted, many that have proven to be failures after long-term use and the research continues 
today. However, when it comes to web-based distance learning environments there appears to be an over 
emphasis on the fact that there are no significant differences between media presentations (Kozma, 1993). 
The research that is driving the development seems to be fed by the designers of web-based course 
management systems and those who are in awe of the technologies and their applications. So what we 
have today is a combination of research on how effective discussion boards are, what you can do with 
simulations, how many people respond in chat or bulletin boards, is there a difference in what students 
contribute, etc. 

Research has shown that computers do leave a lasting impact — a cognitive residue on its users 
(Wijekumar & Jonassen, 2002) and a web-based learning environment may also be expected to do the 
same. This paper begins with an overview of current web-based learning environments and thra reviews 
research studies in the areas of learner variables, tasks, and tools that should encourage more discussion 
and research in web-based distance learning environments. 

Current Trends in Web-Based Distance Learning Environment 



The current crop of web-based course management systems (e.g., WebCT, Blackboard, Angel) all provide 
similar functionality and they appear to drive the content, presentation, interactions, and assessment in 
web-based distance learning environments. 

Research Hig hlights to Inform Distance Learning 

There is some research that has supported the expansion and use of the web but there are some that should 
concern us. The research that supports the use of the web includes research on navigation (Barab et al. 
1998; Montgomery & Stevenson, 1998), collaboration, problem-solving skills using Cognitive Flexibility 
Theory (Spiro et al., 1989), hypertext, and communication. The specific findings of these studies should 
influence our design of wetybased learning environments. The following summary of research is divided 
into areas related to the Learner, Task, and Tools. 

Learner 

Learner variables and individual difference like field independence/dependence, prior knowledge, 
intellectual level, motivation, interest, self-regulation, and ability are critical components for any effective 
learning to occur. Tobias (1989) suggests that computer adaptive learning environments cannot reach 
every type of learner in every type of situation. This su gg ests that learners should be oriented to learning 
on the web and be trained to take address their individual learning needs and self-regulation. 
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Table 1 . A summary of These Tools and How These Tools Affect the Cow-ses. 



Functionality 
of Tool 


Manifestation 
in Course 


Concerns 


Documentation 


Syllabi 


Do the students read the syllabus? 


Linked Web Pages 


Course Notes 


What do they read, when do they read it, and 
how much will they remember, and what are 
they using these notes for? 


Bulletin Boards 


Questions to answer 
Discussions 


Who is posting and what are they posting? Are 
they reading the postings? Are the discussion 
activities relevant to their learning? 


Chat Rooms 


Discussions 


Are these social gatherings or business 
meetings? 


Quizzes 


Timed Multiple 
Choice, T/F Tests 
Essays 


Are these the only methods of assessment that 
are possible and should they be the only means 
of assessment in web-based learning? 


Multimedia and 
other tools 


Pictures, images, 
movies, etc. 


Are these seductive details or relevant to the 
learning? 



Learner control of their learning has shown mixed results and is very troubling. For example, even 
though Cognitive Flexibility Theory has shown improvement in learning of complex skills research has 
shown that students perform well only when they traverse all the links provided in the learning 
environment (Jacobsen, 1993; Jacobsen et al. 1995). In similar studies, students given the control of their 
learning chose not to traverse all the links and their performance was not improved (McKeague, 1996). 

Learner control related to self-regulation and affordances has also provided mixed results. The self- 
regulated learners with good metacognitive skills know when and how to exert there control and monitor 
they learning, many others do not have that skill (Roth, 1 995). 

Task 

Even though hypermedia may be similar in structure to the contents of knowledge structures in humans 
the same media can result in the construction of frail knowledge networks which may be caused by a lack 
of mindful engagement, surfing the web, and interconnected network of nodes (Salomon & Almog, 

1 998). Salomon & Almog described the possible “shallow associationist cognitive networks” that may 
result from students surfing the web and how these “networks would consist of trivial, frail connections, 
having no intellectual merit” (p.235). 

Collaborative learning research is mixed and the most successful collaborative activities are explanation 
giving (Webb, 1995) and argumentation (Wiley & Voss, 1999). Additionally, collaboration requires a 
group of people to adapt to working together as well as accomplishing a goal. 
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Tools 



Seductive details in the form of unrelated explanations in the text that signal the wrong schema for the 
learner, images that take attention away from important information, and other distractions like 
animations may cause more damage to learning (Harp & Mayer, 1998). 



As the selected research reviewed in this paper shows, there are many issues and concerns that are 
not currently being addressed by the web-based course management systems and the designers of web- 
based learning environments. We need to dig a little deeper into the existing research and find more 
information that can inform us to improve web-based distance learning. 
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IntroductioD 

Learners who receive training in the workplace, at their residences, or other sites outside the traditional 
classroom increasingly rely on the use of the World Wide Web. Much of this training occurs in the 
absence of an instructor, both physically and temporally. Reliance upon such asynchronous distributed 
learning systems increases the odds of various forms of training compromise, such as obtaining questions 
beforehand or enlisting a proxy for test taking in non-proctored, Web-based learning environments. How 
can it be determined whether the student online is the intended learner, particularly during individual 
testing? 

To begin to answer this question, a workshop titled “Training on the Web: Identifying, Authenticating, 
and Monitoring Learners” was conducted in November 2001 at Carnegie Mellon University. The 
workshop was sponsored by the Army Training Support Center, which recognized that the increased use 
of computer technology for distribute learning systems might lead to future problems with online 
training and testing. There currently is no definitive evidence that compromise during online testing is a 
problem in the Army. However, the increased use of distributed learning coupled with reports of 
increased frequency of cheating among high school students (DeWan, 1994; McCabe, 2001) and college 
students (Argetsinger, 2001) is reason for concern. Most of the information gathered during the workshop 
is not specific to the military — it can also apply to educational and corporate settings. This paper presents 
a synopsis of the findings and recommendations. 

Methodology 

The workshop included presentations by experts and concluded with a brainstorming session during 
which the participants generated potential solutions. Thirty-one individuals from industry, academia, and 
government attended. The presentations covered the following topics: a) training and testing design, 
where a course and its assessment can be designed to diminish the possibility of cheating; b) test security, 
explaining how the Educational Testing Service prevents and detects cheating; c) Public Key 
Infrastructure (PKI), a system to authenticate and secure transmission of information across the Internet 
using asymmetric encryption; d) biometrics, the process of identifying people based on their physical, 
personal, and/or behavioral characteristics; and e) militaiy legal issues regarding training compromise, 
where cheating is a failure to follow a rule or regulation and, therefore, can be treated as misconduct. 

After the workshop, an advisory panel met to discuss final recommendations. 
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Findings and Reconunendations 



Based on the presentations and discussions, recommendations were developed which are presented here 
in no particular order of merit. The recommendations are meant to function as general guidelines. The 
usefulness of any particular solution depends on the training situation. For example, the use of biometrics 
for identifying a learner might be useful for a course that covers sensitive material or courses that lead to 
some form of certification, but it may be an excessive measure for a course that is merely a refresher of 
basic general knowledge. Also, there may be privacy concerns and policies specific to an organization 
regarding the encoding and storage of individual biometric data. 

A£Brmative Obligations 

Affirmative obligation statements involve presenting a statement that details appropriate and 
inappropriate behavior during a test. Learners are required to sign the statement or click on a “submit” or 
“accept” button on the monitor, as in the generic example below. Affirmative obligations assist in 
establishing that a transgression occurred. 



In accordance with (cite authoritative reference), I certify that the 
answers I submit are the result of my own work and that I have not 
had access to copies of answer sheets or solutions firom others. 

Pressing the submit button to process your examination is your 
assertion that the above statement is true. 





Figure 1. A generic affirmative obligation statement adapted from the U. S. Army. 

Biometrics 

The Defense Biometrics Management Office described the four steps in the process of using biometrics 
for identification; capture, process, em-oll, verify. The capture process is where the device obtains the 
biometric data (e.g., fingerprint, iris image, handwriting sample). The data are then processed and 
encoded to an easily storable form. This encoded data can also be encrypted to provide a higher level of 
security. The enrolling procedure occurs the first time an individual’s biometric data are obtained and 
stored. The verification procedure occurs when an individual’s biometric data are compared to stored 
data to determine if a match has occurred. The technology was demonstrated during the workshop. 

There are some concerns with biometrics. For example, the technologies are not completely foolproof 
There are two types of errors that can be made. The first is a false acceptance, when the biometric data of 
a confederate is accepted. The second is a false rejection, when the correct person is rejected as not 
matching his/her biometric profile. To lessen the chance for these errors, biometrics can be used in 
conjimction with a password, or more than one biometric can be used at a time to identify a person. The 
most common and promising biometric technologies are fingerprints, iris patterns, facial composition, and 
handwriting. 
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Fingerprint identification. This biometric identifies between 15 and 20 distinct minutia points (e.g., 
end of a ridge joining of two ridges) in a single fingerprint, and then the distance and angle between the 
key points are measured. The fingerprint is thereby encoded as vector measurements between these 
points. The actual fingerprint is not stored but rather the polygon that connects the distinct characteristics 
of a fingerprint; thus, the fingerprint image cannot be duplicated fi’om the encoded set of vector 
measurements, increasing security. 

Iris patterns. Scanning a person’s iris (the colored part of the eye surrounding the pupil) is a method that 
is promising, and may be a preferred method in the fiiture. In this procedure, the pattern of radial marks 
and their relative position in the iris are obtained. The iris pattern remains identifiable, even though the 
size of the iris changes as the pupil changes due to lighting. 

Facial composition. The facial composition method identifies points on the face (i.e., eyes, nose, mouth) 
and then measures the relative distance between these key points. The relative distance and angles 
between these facial components can be encoded in a similar manner as fingerprints and iris patterns. 

Handwriting analysis. With current technology, handwriting analysis involves more than just the visual 
shape of the signature. The speed and pressure used to create the signature are measured in addition to 
the shape of the signature. The inclusion of speed and pressure make this biometric much more difficult 
to forge than just the visual aspects of a signature. 

There are some other forms of biometrics, such as retinal images, hand geometry, vein pattern on the back 
of the hand, and ear shape. The technology for these forms of biometrics either are still emerging and/or 
may not be used due to the invasive nature of the measure. Cost is also a factor; to secure a computer 
with a fingerprint system can cost approximately $100 and an iris scanning system approximately $250. 
With further development, these prices are likely to fall in the future. Of the various biometric 
technologies, fingerprint scanning is the most mature method based on cost, reliability, and usability. 
During the workshop, the Center for Identification Technology Research, sponsored by the National 
Science Foundation, presented on future biometric technology, such as perspiration patterns, template 
aging, and body odor. 




Figure 2. Fingerprints fi’om the same finger, but taken at different angles. The lines link corresponding 
minutia points between the fingerprints. 
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Proctoring 



There are two types of proctoring: live and virtual. Live proctoring requires that students go to a specific 
location for monitoring by another person; virtual proctoring involves using technology to monitor 
students at remote locations. 

Live proctoring is expensive but hard to beat for high stakes testing situations where the consequences of 
cheating and not knowing material are severe. For example, it is imperative that a helicopter mechanic is 
aptly qualified to identify and repair faulty parts. There are several companies that provide live 
proctoring services. 

For virtual proctoring, the advisory panel recommends using a layered approach depending on the critical 
nature of the test. Examples of how this layered approach may work are listed below. 

*t* With high stakes tests, video monitoring and a biometric measure such as iris scanning may be 
used. 

❖ For medium stakes tests, a single biometric measure may be acceptable. 

❖ For low stakes tests, no proctoring measures may be needed. 

Alternatively, or in addition to biometrics, students can be asked biographical questions (e.g., last 4 digits 
of social security number or mother’s maiden name) during testing to help verify their presence. The final 
virtual proctoring recommendation is to track keystrokes and the web sites visited by the test taker. This 
can provide evidence if cheating occurs, and can also serve as a deterrent. 

Public Key Infrastructure 

The use of PKI limits unauthorized access to tests and assures that materials are not altered en route. As 
PKI encryption becomes more common, the ease of incorporating the technology into distance learning 
courses will increase. 

Test design 

The principal test design recommendation is to use performance-based testing, where the student must 
demonstrate performance proficiency. Successful completion of a performance-based test indicates that 
the test taker is able to perform the task, and pre-knowledge of test content becomes almost irrelevant. 
Other recommended design techniques are to randomize test items and use multiple forms. This reduces 
the utility of answer keys from previous tests. 

The final set of test design recommendations involves setting limits. First, appropriate time limits can be 
set so test takers do not have time to complete the test plus look up answers in reference materials. 
Additional limits include restricting the number of times that a student can take a test and disabling 
computer “print/capture” options to reduce the possibilities of sharing test items with others. 

Conclusion 

It was the advisory panel’s basic assumption and belief that learners generally will “do the right thing.” 
The solutions, however, are meant to level the playing field, dissuading potential cheaters while not 
burdening those who never intend to cheat. The purpose of the study was to identify remedies to 
compromise in Web-based training and testing environments that can be implemented without hindering 
learning and prior to any problems arising 
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The overall recommendation is a layered approach based on the criticality of the test unda- considwation. 
In some situations the use of biometrics, live proctoring, and encryption may be warranted, while in othw 
situations only the use of an affumative obligation statement may be appropriate. The level of swurity 
would depend on multiple variables determined by a course administrator and organizational policy. 
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Introduction 



Background of the Problem 

Instructional design is aimed at improving instruction to ensure the learning of the individual. 
“Systematically designed instruction can greatly affect individual human development” (Gagne, Briggs & 
Wager, 1992, p. 5). Teaching at a distance, whether synchronous or asynchronous, requires the designer 
to place greater stress on the initial planning phase. Although distance education has become a viable 
alternative or addition to traditional education, there is little research on the instructor and student 
perceptions of the design of the instruction when the two major distance education technologies, the 
Internet and Interactive Television (ITV) are used. Therefore, additional research in this area has become 
the focus of this study. 

Purpose of the Study 

This study examined the perceptions of both instructors and students at the University of Northern Iowa, 
Cedar Falls, Iowa on the need for the instructional design process used when designing instruction for two 
major distance education technologies-- the Internet and ITV. 

Significance of the Study 

It is important to gain instructor and student perceptions/attitude on the design of instruction for the 
Internet and ITV. The information gained from this study provides an understanding of how the use of the 
Internet and ITV affects the design for instruction in distance education courses. The data acquired from 
the study provides administrators and instructors who are involved in distance education with decision- 
making information related to instructional design to enhance the distance teaching process. In addition, 
this knowledge can be used for the distance education program planners and educators to broaden the 
scope of effective learning experiences available to distant learners and to improve the quality of the 
learning experiences of the distant students. 

Theoretical Background 

There is very limited literature resources that address the issue regarding instructor and student 
perceptions of designing instruction related to using the Internet and ITV in distance education. 

Teaching at a distance requires greater stress on the initial planning phase (Simonson, Smaldino, 
Albright, 8 l Zvacek, 2000). To secure the success of the planning, distance learning faculty should pay 
attention to some key issues regarding designing instruction at a distance. 

According to Simonson, et al. (2000) the following issues need to be considered: 
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1 . Who are the learners? The instructors need to have knowledge of general learner characteristics. This 
knowledge about the learners can help the instructor handle the separation of instructor and students 
successfully. 

2. What is the essential content? The content of a course should reflect where this content relates to the 
rest of the curriculum. Instructors need to consider the nature of the content, and the sequence of 
information. Generally, the scope of the content for a course needs to be sufficient to ensure that the 
learning experience will result in desired outcomes. The identification of goals and objectives for 
instruction is necessary. 

3. What teaching strategies and media should be used? Learner participation is important when distance 
educators decide which strategy or strategies to use. The instructor needs to think about selecting 
those instructional strategies that enable all the learners to engage actively in their own learning. 

4. What is the learning environment? To completely understand distance education, one must examine 
not only the technology, but the learning environments that are created (Herring, & Smaldino, 1998). 

5. How do you determine the quality of the instruction? Since so many different variables can influence 
the effectiveness of the quality of instruction, it is important that the instructor reflect on distance 
teaching practices. Formative evaluation is necessary for successful distance teaching/leaming 
experiences. 

6. The distance education instructor needs to address general learner characteristics, the nature of 
content, teaching strategies and media selected, the learning environment and evaluation. These are 
important factors related to designing instruction at a distance. Based on those factors, the study 
described in this paper reports on the perceptions of both instructors and students on the design of 
instruction when conducted via the Internet and ITV by focusing on four factors or areas: (1) the 
Internet and interactive television technology characteristics, (2) content organization, (3) learner 
considerations, (4) evaluation. 

Statement of Research Questions 

Surrounding those four areas, research questions are generated as follows: 

1 . What are UNI distance teaching instructor and distance learning student perceptions toward 
technology characteristics, content organization, learner considerations, and evaluation? 

2. Is there any significant difference in perceptions of technology characteristics 

3. between the UNI distance teaching instructors and distance learning students involved in teaching or 
learning the courses offered at UNI? 

4. Is there any significant difference in perceptions of content organization 

5. between the UNI distance teaching instructors and distance learning students involved in teaching or 
learning the courses offered at UNI? 

6. Is there any significant difference in perceptions of learner considerations 

7. between the UNI distance teaching instructors and distance learning students involved in teaching or 
learning the courses offered at UNI? 

8. Is there any significant difference in perceptions of evaluation between the UNI distance teaching 
instructors and distance learning students involved in teaching or learning the courses offered at 
UNI? 
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Methodology 



Participants 

Eight instructors with ITV and Internet teaching experience and eighteen distance students enrolled in an 
ITV/web course were chosen to participate in this study. A mailed self-completion survey was distributed 
along with an introductory letter attached to the questionnaire. A total of seventeen students and six 
instructors returned their questionnaires by the deadline date. Among the six instructors, four were from 
the Department of Curriculum and Instruction, one from the Department of Educational Psychology and 
Foundations, and one from the Department of Science Education. The seventeen distance students were 
all from one graduate level Instructional Design course. Among the 23 returned surveys, three were not 
used in the study because one instructor only used ITV and two students’ surveys contained missing 
important demographic data. 

Instruments 

The survey used in this study consists of two questionnaires, one for instructors and other for students. 
The two questionnaires were generally composed in the same way: both containing 26 items in which the 
same issues are addressed (except for in 23 and 24 items). The questions found in the two survey 
instruments were developed from issues and questions arising from the initial review of literature. The 
research questions served as guideposts in developing the specific items. 

The four areas investigated in the questionnaires are identified as follows: the Internet and Interactive 
television technology characteristics being included in the first ten questions; content organization being 
identified by nine questions; learner considerations being defined in three questions; evaluation being 
covered by two questions. Two open-ended questions asking about the major strengths and weaknesses of 
using the Internet and ITV for teaching and learning the content are used to provide some detailed 
information to complement the UNI instructor and student perceptions related to those four areas 
investigated. 

Validation of the instrument. Before doing this study, a focus group was used to test the research 
instrument to provide another source of information and a second data base. This group consists of one 
professor and five students in a research course in Fall, 2000. Since surveys are expensive and time- 
consuming, it is important to ask the right questions and to ask them in the right way. By conducting this 
focus group before initiating a survey, the researcher could formulate the survey questions more precisely. 
This focus group also helped the researcher to understand the prospective respondents’ general 
perspective on the issues to be discussed, frame of reference to be used, and way of thinking and typical 
vocabulary when handing the topics at hand. The researcher applied the input provided by this focus 
group to the construction and revision of the two questionnaires. 

In this study, a five-point Likert scale ranging from strongly agree to strongly disagree was used for the 
participants to answer the structured questions of the two questionnaires. Open-ended questions were also 
included in the questionnaires for more information about perceptions of strengths and limitations of the 
technologies. 

Procedure 

The five distance instructors and 15 distance students at UNI who participated in the study were given 
two weeks to complete the questionnaire and returned it. 
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Data Analysis 

Both quantitative and qualitative methods were used to analyze the data. The data regarding the structured 
questions in the two questionnaires was statistically analyzed using SPSS 9.0 statistical software package. 
Means and standard deviations were calculated for each statement on the survey instrument. In addition, 
an overall cluster mean was calculated for each of the four cluster areas. An independent-samples t-test 
analysis was conducted on each of the four cluster areas to determine if there was a statistically 
significant difference that existed between the instructors’ and students’ perceptions of using the Internet 
and ITV for the design of instruction. An alpha level of .05 was used to determine significance. A coding 
system which a qualitative study usually applies was used to analyze the data regarding the open-ended 
questions in the two questionnaires which discuss the major strengths and weaknesses of using the 
Internet and ITV for teaching and learning the content. Some important themes were generated. 

Results 

Research Question 1 intended to determine the overall group means for each cluster of questions (see 
Table 1). The results based on mean scores among the two groups dealt with four clusters: technology 
characteristics, content organization, learner considerations, and evaluation. Generally, student cluster 
mean scores were rated lower than instructor cluster mean scores. 

Research Questions 2, 3, 4 and 5 intended to determine if the differences found for the four cluster area 
means between two groups were significant at the .05 level. An independent-samples t-test revealed that 
the student group rated statements significantly lower than did the instructor group (see Table 2). 
Significant differences were not found among the Internet and Interactive television technology 
characteristics, content organization and learner considerations clusters between instructors and students. 
However, significant difference was found among the evaluation cluster between the two groups. 

Discussion 

The findings of this study showed that although the student group rated the four clusters of statements 
lower than did the instructor group, they still had generally positive perceptions of the design of 
instruction related to the four areas investigated in this study. The general themes generated through 
analyzing open-ended questions are presented as follows. 

The distance instructors and students provided some major strengths of using the Internet and ITV for 
teaching and learning the content. They generally believe that the Internet and ITV technologies 
complement one another very well and provide good educational experience. Both instructors and 
students agreed that obviously the use of the two technologies overcame limitations of time and distance 
because it allowed students to interact outside of class and made courses available to those who otherwise 
could not come to campus to take them so that more students can attend class. Instructors could also reach 
more students who took interest in their programs. Students commonly believed that taking ITV classes 
saved time traveling to campus and they could work and go to school at same time. Regarding the 
Internet-based courses, students generally observed that WebCT was an excellent user-fi'iendly tool for it 
definitely helped them master the content. They believed that it had expanded resources and provided up- 
to-date information that print articles would otherwise carry and the schedule for it was flexible. 
Communication via e-mail/bulletin board was easy, friendly, and quick. They could work on WebCT at 
any time that was convenient to them especially when their schedule is very tight. And this makes 
discussion more interactive. Concerning ITV classes, some students believed that since the schedule for 
ITV classes was more structured, and it allowed time for group discussion. And it was also good for 
students to promote camaraderie among themselves. Some students felt that distance learning via the two 
technologies was challenging and student-centered. Students could communicate the content with 
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distance learners from different locations with a variety of backgrounds and focuses and on-site 
professionals with a wide variety of perspectives across the state. 

The distance instructors and students also offered some major weaknesses of using the Internet and ITV 
for teaching and learning the content. The major concern for ITV is the technical difficulties for those at 
distance sites. Quite a number of instructors and students reported that technical problems had been 
frequent on the ICN which was frustrating. Having a personal contact could solve this kind of problem 
and establish great support for students. 

Lack of interaction and sharing of ideas is another concern for the Internet and ICN teaching. Both 
instructors and students felt that face-to-face interaction with the professor and other students was not 
possible on the ITV. Often there were not much small group discussion and activity. Sound was 
sometimes a problem. All the facilities were not equal in terms of resources for sharing information. 

Some instructors felt that sometimes they were lost in not being able to connect with their students on a 
personal level. Students also had the same feeling of losing one-on-one interaction with the teacher and 
other students. Sometimes, they felt that classes were too large for interaction; thus, a large part of class 
became teacher-centered. Some students indicated that they were shy to be on ITV, which may have 
contributed to limited interaction. Regarding the Internet-based instruction, students felt that they had to 
accept ideas as they were. The use of the Internet was somewhat of an impersonal experience. In addition, 
students commonly felt the Internet resources were too vast and it was easy to lose focus. To them, the 
Internet was a great resource to obtain information; however, it was hard to find exactly what they needed 
sometimes. Besides, some information was irrelevant. Finally, students felt that the ITV teaching lacked 
spontaneity because this kind of teaching was very much structured and that lecture format dominated the 
class though they knew that lecture was the best way to transfer information under these circumstances. 
The findings of this study have indicated needed areas for improvement in the distance learning program 
at UNI. Specifically, strategies need to be developed to help distance instructors and learners to 
overcome technical difficulties, additional measures need to be taken to enhance instructor-student and 
student-student interaction, such as small group discussion, presentations and projects and to help 
students to find the resources quickly which directly satisfy their learning needs. 



Table 1. Overall Cluster Item Means for Each Cluster Group 





Instructor 

Mean 


Student 

Mean 


Total 

Mean 


Technology Characteristics 
Overall Cluster Mean 


2.2400 


1.9167 


2.0118 


Content Organization 
Overall Cluster Mean 


1.8889 


1.6370 


1.6901 


Learner Consideration 
Overall Cluster Mean 


1.9167 


1.4667 


1.5614 


Evaluation 

Overall Cluster Mean 


2.0000 


1.2000 


1.4000 
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Table 2. Independent-Samples t-Tests for Technology Characteristics, Content Organization, Learner 
Consideration and Evaluation Cluster 





Mean 


SD 


t- 

Statistic 


P 

Value 


Technology Characteristics 
Instructor 
Student 


2.2400 

1.9167 


.33615 

.31861 


-1.878 


.108 


Content Organization 
Instructor 
Student 


1.8889 

1.6370 


.46259 

.42718 


-1.032 


.376 


Learner Consideration 
Instructor 
Student 


1.9167 

1.4667 


.87665 

.50079 


-1.367 


.387 


Evaluation 

Instructor 

Student 


2.0000 

1.2000 


.36839 

.35355 


-4.243 


.003* 



* Indicates a significant difference at .05. 
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Introduction 

Online learning has matured to the point where most people are worrying not only about how they will do 
it, but also how th^ will assess and evaluate it. Clarity is needed for the online instructor — and those 
concerned Avith its successes and failures — in each of these areas. This paper reviews some of the 
pedagogical practices that we recommend for online learning, ideas for assessment, and finally some 
considerations for evaluating online courses. 

Pedagogical Practices and Learner Assessment 

Each online course is unique, with specific design and implementation being dependent on myriad factors 
including subject matter area, learning objectives, learner characteristics, instructor characteristics, and 
technological/environmental factors. It is the instructor or course designer’s task to analyze these factors 
and select and adapt appropriate activities accordingly. Table 1 presents a review of some pedagogical 
activities that might be tried, as appropriate to the content area, in an online course. Each activity serves a 
particular learning purpose and can be used to meet different types of course objectives. 

These activities can be conducted via synchronous or asynchronous communication tools, with each 
having unique characteristics and challenges. Overall advantages of synchronous activities are the 
spontaneity of communicating live, particularly when immediate feedback or response is desirable, and 
the increased likelihood that each contribution will get a response. Challenges include an increased need 
for structure and facilitation, especially with respect to tum-taking, and the lack of reflective participation. 
Online learning activities that are conducted asynchronously benefit from the ability of students to reflect 
on the discussion and carefully compose their responses. Some of the biggest struggles of using an 
asynchronous medium for learning activities is ensuring that students now how, how often, and when to 
respond to each other. If not carefully planned, activities can result in a series of isolated responses rather 
than true learning dialogues with students engage in message-posting rather than discussion-oriented 
behavior. Table 1 provides an overview of a few of the online learning activities that we encourage 
instructors to try, along with some differences to be considered based on the communication medium 
being used. 

Assessment is an important component of online learning activities. While many in-class activities may 
not be assessed in any formal manner or only under the guise of a generic “class participation” grade, 
students generally tend to participate so long as they are attending the class. Essentially, if they are 
present, they may as well be doing what is asked of them. The online instructor has the added challenge 
of getting students to be present and visible in the course space. Providing some form of assessment of all 
course activities and letting students know that the messages they generate will be graded is an important 
way of providing them with incentive to be active learners. They are not a captive audience, and their 
participation in online learning activities often requires greats engagement and effort than similar 
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participation in a classroom. Since the students are generating text-based records that can be reviewed for 
content and quality, letting them know that someone is reading their contributions and whether or not 
their work is of an appropriate quality and quantity is important. 

A variety of assessment techniques may be used, varying in amount of feedback provided to students and 
labor-intensity on the behalf of the instructor. It is not necessary for instructors to provide lengthy, 
detailed comments to individual students in a large class; trying to do so is one of the things that is very 
likely to overwhelm and exhaust the instructor. Posting general feedback to the class at the end of each 
activity, pointing out students with excellent work by name, is an effective way of maintaining 
participation and quality. Rubrics can be used to give quick indicators of performance and assign grades. 
Students can provide each other with regular feedback by assigning them buddies to comment on each 
other’s work on a regular basis. 



Evaluation Techniques 

The evaluation of online learning is important to many people. For instructors, it allows them to 
determine what is working and what should be changed. Potential students may wish to know how highly 
rated or effective a particular course is before choosing to enroll. Administrators want to know not only 
that an instructor is performing well and that students are learning, but also that the resources they have 
devoted to developing and delivering the course have provided a sufficient return on investment. 

This evaluation can occur in many ways and at many levels. Instructors should document materials and 
assignments that worked well and did not work well so that such impressions can be triangulated with the 
students’ learning results and student feedback in the formative interest of making improvements for the 
next course offering. Care must be taken to ensure that appropriate end-of-course evaluation questions are 
asked of students; evaluation forms from classroom-based courses may ask students to evaluate the 
instructor and the learning experience in ways that do not make sense (e.g. an instructor’s delivery style) 
while ignoring important elements (e.g. how well the technology worked and an instructor’s presence). In 
some instances subtle wording changes on a traditional evaluation form can change a student reaction 
from thinking “this was not applicable to this class,” to being able to make an evaluative judgment. 

At the curricular level, it may be important to evaluate whether or not an online course is adequately 
preparing learners to perform particular tasks in higher-level classes (or, in the case of online training, on 
the job). Doing so is a larger-scale evaluation endeavor than is typically undertaken, but it can be very 
worthwhile given the extensive resources that are often required to design and develop a new online 
course. Evaluation data may be collected by following up with learners at a later date as well as the 
instructors of subsequent courses (or job supervisors and colleagues) to determine if the learners are 
choosing to and able to successfully apply what they learned in the online course. 

Closing 

In summary, it is the combination of appropriate pedagogical, assessment and evaluation techniques that 
will allow online learning to flourish and become more successful for instructors, students, and other 
stakeholders. We have presented some ideas to get people started on the process of developing good 
practices in each of these areas, but even more importantly we encourage others to continue to 
experiment, reflect and share their results to help build the body of knowledge of best practices in online 
learning. 
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Table 1. Sample Online Learning Activities 



Type of 
Activity 


When to Use 


General 
Structure and 
Elements 


Synchronous 

Issues 


Asynchronous 

Issues 


Learner 

Assessment 


Ice 

Breakers 


When you want 
learners to get to 
know each other 
When you want 
to introduce a 
new topic 
(essentially, 
“ice-breaking” 
the topic) 


Everyone gets 
an opportunity 
to share or 
participate; 
wide variety of 
potential 
activity 
frameworks, 
including Two 
Truths and One 
Lie, Coffee 
House 

Expectations, 
20 Questions 


Difficult to hear 
from everyone 
(turn-taking is 
necessary); 
certain activity 
frameworks 
will work better 
than others 


Learners may 
only selectively 
participate and 
read messages; 
effort must be 
taken to 
encourage them 
to “meet” all 
classmates or 
read all 
messages in a 
new topic ice 
breaker 


Assessment is 
generally not 
necessary, but 
you’ll want to 
make sure 
everyone has 
participated 


Role 

Play 


When you want 
learners to see 
the multiple 
perspectives on a 
particular topic 
or to view a 
topic through the 
lens of different 
experts/theorists 


Learners are 
assigned a role 
to play (role 
may be a 
perspective — 
like optimist, 
pessimist, 
journalist — or 
a person — 
like Kant, 
Nietzsche, 
Foucault) 


Learners must 
fully 

understand their 
roles in order to 
be able to play 
them out in real 
time; Some 
form of turn- 
taking must be 
in place to 
ensure that all 
participants can 
be active 


Learners must 
have 

participation 
guidelines and 
deadlines to 
ensure that 
dialogue takes 
place; 

summarization 
of discussion is 
important to 
bring closure, 
and effort must 
be taken to 
encourage 
learners to read 
the summaries 


Learners can be 
assessed on the 
nature and 
quality of their 
participation 
(How much did 
they contribute? 
How well did 
they represent 
their assigned 
role) or the 
exercise can be 
used as a learning 
tool working 
toward a larger 
assessment like a 
paper or project 
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Table 1. Sample Online Learning Activities (Con’t) 



Type of 
Activity 


When to Use 


General 
Structure and 
Elements 


Synchronous 

Issues 


Asynchronous 

Issues 


Learner 

Assessment 


Debates 


When you want 
learners to 
research and 
focus in-depth 
on two 

perspectives of a 
potentially 
controversial 
topic 


Learners may 
be assigned a 
topic and a 
side, either as 
an individual 
or group, and 
given time to 
research the 
topic and 
prepare 


Turn-taking 
must be 
carefully 
facilitated to 
ensure 
equality for 
both sides 
and all 

members of a 
group 


Timing must 
be carefully 
structured to 
allow for 
dialogic 
interchange 
between sides; 
rebuttals 
should be 
deeper and 
more 

reflective than 
in a 

synchronous 
debate, with 
resources 
cited 


Learners can 
be assessed 
on their 
topical 
research, 
development 
of arguments 
and rebuttal 
arguments; 
additionally 
the exercise 
can be used 
as a formative 
assessment of 
learner ability 


Peer 

Feedback 


When you want 
learners to help 
each other 
improve their 
work, learn to 
evaluate and 
critique, and/or 
practice using 
the terminology 
of the field with 
respect to 
student-produced 
documents 


Learners are 
asked to review 
and comment 
on each other’s 
idea and work; 
rubrics may be 
provided to 
help students 
focus on 
appropriate 
criteria 


Students 
providing 
feedback 
must review 
material in 
advance and 
be prepared; 
students 
receiving 
feedback 
benefit from 
ability to 
seek 

clarification 
of muddy 
points in real 
time; 

important to 
have way of 
saving 
feedback for 
later use 


Encourages 
more hi^ly 
reflective 
feedback than 
synchronous 
feedback 
sessions; 
timing of 
making work 
available for 
critique and 
providing 
feedback is 
important; 
may wish to 
allow students 
receiving 
feedback time 
to ask their 
respondents 
for 

clarification 


This activity 
should 
provide 
learners with 
formative 
assessment of 
their own 
work. 
Potential 
summative 
assessment 
areas include 
quality of 
feedback 
provided to 
peers and 
improvements 
made to own 
work based 
on peer 
feedback. 



O 

ERIC 
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Abstract 

As e-leanting becomes increasingly popular, comprehensive evaluation is critical to quality improvement. 
This empirical study was based on a comprehensive review of literature and subsequent development of a 
multidimensional evaluation matrix. The matrix ’s components are six factors plotted against 5 subjects; 
each of the 30 cells is then categorized into evaluation levels of reaction, learning, behavior, and results. 
Factors include learner characteristics, technology, instruction, instructor, institution, and community, 
while subjects include learners, instructors, support staff, administration, and community. The matrix 
provides a flexible and holistic means to evaluate e-leaming. 

Introduction 

According to the American Society for Training and Development (ASTD), e-leaming refers to “anything 
delivered, enabled, or mediated by electronic technology for the explicit purpose of learning” (Tanquist, 
2000). Online programs are increasing in number and becoming one of the most important new trends in 
education (Williams, 2001; Kirk, 2001). There has been a 72 percent increase in the number of distance 
education programs at higher education institutions (National Center for Education Statistics, [NCES], 
1999). With the continuing growth of university-level courses offered at a distance, researchers are 
focusing their attention on the issue of program evaluation (Biner, 1993). However, “e-leaming 
evaluations often do not receive the priority that they deserve” (Tanquist, 2000, p. 1). Although there has 
been limited research on how to evaluate DE courses in general, there has been little attention given to a 
more comprehensive evaluation of e-learaing. 

This research illustrates a comprehensive evaluation process. It highlights the reasons why it is cmcial to 
evaluate e-leaming, the evaluation process, subjects who should be involved, and the major factors and 
variables that need to be included in an evaluation instmment. 

Research Questions 

The research questions guiding this study were: 

1 . What is the value of evaluating e-leaming courses and programs? 

2. What is the process of evaluating e-leaming courses and programs? 

3. Who should be involved in e-leaming evaluations? 

4. What are the variables that need to be included in e-leaming evaluation measures? 

Value of Evaluating E-leaming 

E-leaming is relatively new and is ridden with skepticism, caution, and resistance and one way to 
suppress such concerns about e-leaming is through evaluation (Tanquist, 2000). Magalhaes and Schiel 
(1997) cited a lack of quality control in distance and traditional systems as one of their shortcomings and 
emphasize the need for ongoing evaluations. It helps to identify areas of a program that are producing 
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negative reactions (Biner, 1993); to provide input that could help to improve the quality of the programs 
from their inception to implementation (Magalhaes & Schiel, 1997); and to analyze the strengths and 
weaknesses of a program (Foxon, 1989; Weigand, 2000). Developing and implementing online programs 
is a large investment hence the need to evaluate its effectiveness in terms of client satisfaction and 
economic return is apparent (Tanquist, 2000). If clients, in business in particular do not perceive a return 
on investment in tangible or intangible benefits, they may not be willing to continue to invest in the 
instruction (Boverie, Mulcahy, & Zondlo, 1994). Online learning not only involves the instructor-learner 
interaction but also other facets like learner-environment, learner-technology, and learner-learner 
interactions. Evaluation instruments should focus on variables within and outside the online classroom. 
Therefore, an evaluation of e-leaming needs to be examined from a systems point of view as reflected in 
the matrix. 



Research Method 

This study was based on an extensive review of evaluation literature. Concepts commonly used in e- 
leaming evaluation, evaluation variables, subjects or sources of data, and data collection procedures 
emerged. In addition to literature reviews, faculty and staff members were contacted for feedback on how 
to improve the developed instrument. The authors synthesized this information into a matrix (Table 1 - 
The Hatcher-Mungania E-leaming Evaluation Matrix), which is used in the development of e-leaming 
evaluation measures. 



Phases of Evaluation 

Based on the review of literature three types of evaluation are recommended; pre-course, formative (in- 
process), and summative evaluation. A pre-course survey or questionnaire is administered at the 
beginning of a course of study. It seeks demographic data from the learners, their characteristics, 
preparations they have made for the online course, and helps to identify any learner concerns before the 
course begins. Formative evaluation is used during instmction to identify aspects of the program that the 
participants consider successful or unsuccessful (Magalhaes and Schiel, 1 997) and seeks feedback on any 
factors that may affect the teaching- learning process. To collect formative data, individual or group 
journals, submitted assignments, chats, audit trails, discussion fomms, surveys or questionnaires are used. 
Summative evaluation is administered towards the end of the semester or after the course (post-course) to 
learners, instmctor(s), administrative staff, support staff, and to community members. Summative data is 
obtained from surveys, questionnaires, focus group interviews, face-to-face interviews, or through group 
projects, and from individual or group final examination results. The aim of summative evaluation is to 
justify the implementation of an instructional technology, or to help verify whether or not the difficulties 
experienced at the formative level were resolved (Magalhaes & Schiel, 1997). Changes could be made in 
the program based on such information. 

Pilot evaluation.' Thyer, Polk and Gaudin (1997) suggest caution prior to widespread adoption of DE 
instructional media and the authors also recommend piloting evaluation instruments before undertaking 
an institution/organization-wide evaluation. The instrument developed as a result of this study is being 
piloted with online courses at the University of Louisville. 

Subjects 

Because e-leaming is a collaborative effort the subjects for an evaluation include (1) learners enrolled in 
an online course or students who have dropped the course; (2) course instructor; (3) support staff like 
instructional designers, teaching assistants, and technical support staff; (4) administrative staff 
representing the institution; and (5) community members. These subjects are the source of data pertaining 
to various aspects of online learning. These five categories of subjects are labeled under columns A 
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through E as indicated in Table 1 . A data collection tool such as a survey is administered to each of these 
subjects to find out what their experiences are with regard to the six evaluation factors described next. 

Evaluation Variables 

The matrix consists of six evaluation factors: (1) learner characteristics, (2) technology, (3) instruction, 

(4) instructor, (5) institution, and (6) community’s involvement. These factors are labeled A1 - A6. 

Within each of these factors are several evaluation variables that need to be included in any evaluation 
instrument. Due to space limitations, only part of the variables (Al-El) are discussed to give a general 
idea of the kinds of variables that can be included in an evaluation instrument. The choice of the variables 
to include in an instrument would depend on the purpose of the evaluation and available resources. 

Evaluation Factors and Variables 

Learner characteristics (A1 - El): Learner characteristics important to measure include; demographics, 
the learners’ background knowledge, personality characteristics, expectations, communication 
competency, language level, keyboard skills, academic standing, previous online experience, familiarity 
with technology, time management, motivation level, study habits, learning preferences, and attitude. 

Technology (A2 - E2): Technology is the most significant difference between traditional evaluation 
principles and e-leaming evaluations (Tanquist, 2000). Aspects that could be integrated into closed-ended 
questions include: access to technology, technical support, homepage and content layout, ease of use, 
technical training, software and hardware used, cost of technology, and infi-astructure. 

Instruction (A3 - E3): Before much time and money are invested in e-leaming, evaluation should be 
done during the development of materials (Driscoll, 1998). Some of the evaluation variables under this 
factor are content organization, quality, accuracy, completeness, difficulty, workload, time requirements, 
effectiveness of instmction, interaction, feedback, adult learning principles, relatedness of content to 
goals, and quality of instructional design. 

Instructor (A4 -E4): The success of DE rests on the role played by faculty. A survey or focus group with 
instructors and students may be used to measure instructor effectiveness, communication skills, feedback 
quality and fi-equency, communication skills, content knowledge, attitudes, motivation, challenges, 
access, and interactions. 

Institution (A5-E5): The institution needs to make arrangements for services such as registration, library, 
bookstore, and other student services to be available for e- learners (Wong, 1999). Issues to be addressed 
in an evaluation relate to institutional support, cost effectiveness, efficiency, quality, student 
demographics, cost benefits related to e-leaming, and training available (Magalhaes & Schiel, 1997). 

Community (A6 - E6): By virtue of the fact that online learning is a fairly nevv concept, it is essential to 
know to what extent society as a whole recognizes and supports those who are involved in it. This 
includes student support, recognition of credits for transfer or employment, and Ae esteem to which such 
courses and institutions are held in the community at large (Keegan, 1996). Subjects include employers, 
mentors, subject matter experts, parents, or other stakeholders. Subject matter experts could also be 
instmmental in determining the validity, correctness or completeness of course content (Dnscoll, 1998). 



Sunmiary and Conclusion 

Unlike in traditional educational settings where only the students’ reaction and learning is normally 
evaluated, the matrbt discussed in this study emphasizes the need of a holistic but flexible evaluation 
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process that gathers feedback from various stakeholders such as learners, instructors, support staff, 
administrative staff, and community members on various aspects relating to learner characteristics, 
technology, instructor effectiveness, institutional support, and social support mechanisms in place for this 
new form of teaching and learning. “All e-leaming activities should incorporate at least some form of 
evaluation” (Tanquist, 2000, p. 5). This is reiterated in this study by integrating evaluation tools in every 
stage of the e-leaming process. Most stakeholders will be involved in e- learning in some fashion in the 
future and would greatly benefit from evaluations. 
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Table 1 . Hatcher-Mungania E-learning Evaluation Matrix 



Subjects 







A 


B 


C 


D 


E 


Evaluation Factors & Variables 




Learners 


Instructor 


Support staff 


Administration 


Community 


1 . Learner 
characteristics 


A1 

E-leamers’ 

Demographics, 

Communication 

skills. 

Motivation, 
Attitudes, 
Learning style. 
Personality, 

Prior experiences. 
Cost, 

Technical access 
& Competence, 
Reasons, 

Expectations & 
Preparations done 
for EL 

Evaluation levels: 

Reaction 

Learning 


B1 

E-leamers’ 

Characteristics, 

Motivation, 

Participation, 

Attitude, 

Expectations 

Evaluation 

Levels: 

Reaction Level 
Learning 


Cl 

E-learners’ 

Characteristics 

Motivation, 

Attitude, 

Expectations 

Evaluation levels: 
Reaction Level 


D1 

E-learners’ 
Demographics, 
Attrition rate. 
Retention rate. 
Academic standing, 
Target population 

Evaluation levels: 
Reaction Level 


El 

E-leaming 
graduates’ 
Employability 
Job performance. 

Transfer of learning 
Worth of Credentials, 
Job attendance rates, 
and Expectations 

Evaluation Levels: 

Reaction 

Learning 

Behavior 

Results 




2. 


Technology 


A2 


B2 


C2 


D2 


E2 




3. 


Instruction 


A3 


B3 


C3 


D3 


E3 




4. 


Instructor 


A4 


B4 


C4 


D4 


E4 




5. 


Institution 


A5 


B5 


C5 


D5 


E5 




6. 


Community 


A6 


B6 


C6 


D6 


E6 
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Establishing a Need 

With the advent of instructional technology (e.g., Internet and distance learning), virtual field trips 
substantially reduce or eliminate the logistics (e.g., transportation, permission slips, fees, and chaperones) 
associated with arranging physical field trips (Cooper & Cooper, 1999, 2000). Additionally, access to 
supplemental instruction content located at a prohibitive physical distance from the learner’s home 
schools can widely enhance learning (Griffin & Symington, 1997). Field trips have also been shown to 
promote long-term recall (Falk & Dierking, 1997). As there is typically no commute involved with a 
virtual field trip, virtual field trips enable the learner to have access to and experience a wider range and 
variety of instructional content than those typically provided for in physical field trips. 



Because of the Internet medium of virtual field trips, teachers can now expand and enhance their student’s 
learning beyond the walls of the traditional classroom using this form of technology. Teaching strategies 
will include judging which virtual field trip choices are DAP, how to appropriately integrate the 
information into an early Childhood Education Program (ECED) Program, and methodologies that will 
them to extend their student’s learning after experiencing a virtual field trip. In the process, teachers will 
provide their students with interactive learning experiences that will enable them to construct new 
knowledge, touch upon all students’ specific learning modalities, and enrich their learning in a 
developmentally appropriate manner. 



Theory and Assumptions 

The workshop for early childhood is designed and based on Developmentally Appropriatef Practice 
(DAP) (Bredekamp & Copple, 1997), and supported by the National Association for the Education of 
Young Children (NAEYC) Position Statement on Technology for young children between the ages of 
three and eight years of age. The theoretical basis that supports virtual field trips and the use of 
technology in early childhood education emanates from the work of Seymour Papert and Jean Piaget 
(Haugland & Wright, 1997). The National Science Foundation (NSF) advocates the use of technology in 
Education America for the 2 1 st Century (NSF, 1 983). The NSF further recommends that children both 
learn through and with computers, and that computers be utilized to learn about computers. 

Locating, Evaluating, and Selecting Virtual Field Trip Content Providers 

Generally, locating a virtual field trip and/or content provider can be accomplished in three ways 
textbooks, Internet searches, and searchable databases. Two such texts used by the workshop presenters 
include Gail and Garry Cooper’s Virtual Field Trips and New Virtual Field Trips. Both Cooper texts 
provide subject heading categories which may be of use to educators searching for a particular content 
area such as the Civil War or Marine Life. As new field trips are being frequently developed, Internet 
searches using a search engine (e.g., vvdsenut or teoma) can be utilized to locate specific providers and/or 
instructional content. Lastly, PacBell’s Videoconference Directoiy 

(http://www.kn.pacbell.com/wired/vidconftdirectory.html) or the University of Akron’s K-12 Distance 
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Learning Resources (http://www.uakron.edu/distance/kl2) are two examples of virtual field trip 
searchable databases. Additional information and/or questions may also be directed to the content 
provider, with contact information generally listed on the web site. 



Evaluating the content of a virtual field trip is similar to that of evaluating a textbook for classroom use. 
Typically the content, method of delivery and supplemental print instructional materials (where 
applicable) are under the control of the third party content provider. In some instances, virtual field trip 
content providers may provide access to print materials prior to purchasing the field trip or may have 
demos or previews of individual virtual field trips. As an example, the Cinciimati Zoo web 
(http://www.cincyzoo.org) site contains print materials to accompany their virtual field trips. These 
documents are downloadable in platform non-specific format (i.e., Adobe Acrobat). The Southwestern 
Wisconsin Instructional Network Group (SWING) web site (http://www.swing.kl2.wi.us/) currently has 
links to selected virtual field trip previews using Apple Computer’s QuickTime format 
(http://www.apple.com). Both methodologies allow the classroom instructor to review content and 
delivery methodologies to ensure that the virtual field trips can be effectively integrated into the 
curriculum and meet intended learning objectives. 

For the educator wishing to utilize a model to evaluate virtual field trips, Kirkpatrick (1996) provides a 
Four-Level Model of Evaluation, which serves as a summative assessment of for training programs. 

These levels include 1) reaction, 2) learning, 3) behavior, and 4) results. Student reaction to a particular 
learning program as well as instructor reaction can be used to judge the overall impact of the virtual field 
trip. Learning, defined as “a measure of the knowledge acquired, skills improved, or attitudes changed” 
(Kirkpatrick, 1966, p. 55) can be measured by traditional assessment techniques. In some instances, print 
materials included by some virtual field trip content provides can serve as summative assessment (e.g., 
objective tests). Behavior, typically residing in the affective domain, can be impacted by the virtual field 
trip’s subject area (e.g., multicultural awareness). Lastly, results, as an overall effect on student learning, 
can be ti^ to standardized tests (e.g., Ohio Proficiency). 

Integrating Virtual Field Trips Into K-8 Curriculums 

Physical field trips without direct ties to curriculums and/or learning activities can only be considered as 
forms of entertainment. In a similar fashion, learners can perceive virtual field trips without proper 
integration as a form of media entertainment. While curriculum integration is more than merely showing a 
video during class, lesson plans designed to effectively integrate virtual field trips into the curriculum are 
perceived as one method ensure that the virtual field trip is not merely a from of classroom entertainment. 
It is important to realize that using computer technology with young children is a process of exploration 
and discovery, (Haugland & Wright, 1999). As children progress from kindergarten through the primary 
grades, they not only should be provided with opportunities to make choices about computer experiences 
but, the teacher should also use the computer for more directed activities that match grade-level learning 
objectives as outlined in the educational standards at both the state and local levels. Meeting these 
benchmarks for education in an appropriate manner requires providing children a holistic approach to 
learning through their educational experiences. 



Because young children learn holistically, their learning experiences should be integrated during then- 
early years of development therefore; there is no need for them to distinguish learning categories through 
separate subject areas. The relevant principle of instruction throughout early education is that curriculum 
should be integrated across subject areas. Integration of curriculum is accomplished in a variety of ways. 
Curriculum may be planned around themes or projects based on children’s interests (Bredekamp & 
Copple, 1997). When developing curriculum, all developmental domains of young children should also 
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be taken into consideration. Young children learn through relevant direct experiences and interactions 
within their environment. Therefore, an appropriate classroom environment must be designed and 
equipped with learning centers in which children learn through discovery, exploration, interactions with 
each other, and with hands-on experiences. In today’s world, as computer technology becomes more 
capable and pervades more aspects of society, it is imperative to integrate it into the curriculum. More 
information is deemed important to learn than ever before and the base of essential information grows 
constantly (Roblyer & Edwards, 2000). An integrated curriculum can be enhanced through the use of 
virtual field trips. Appropriately chosen virtual field trips are capable of enhancing students learning by 
providing them with experiences that are relevant, engaging, and meaningful to their personal 
construction of knowledge, (Katz & Chard, 1989). 

Learning Theories 

Educators credit theorists John Dewey, Lev Vygotsky, Jean Piaget, Jerome Brunw, and Seymore Papert 
with some of the fundamental premises of constructivist thinking. The following is a capsulation of each 
theorist’s viewpoint. 

1 . Dewey is known for laying the theoretical groundwork form many characteristics found in today’s 
educational system. He is primarily credited with laying the groundwork for the progressive 
movement in American education. Dewey’s ideas support constructivist models of teaching and 
learning. 

2. Vygotsky’s work on human development and the social cognitive theory along with his twin 
concepts of scaffolding and the zone of proximal development are important for constructivists. 

3. Piaget in internationally known for his complex theories on cognitive development in children. 

Piaget believed that through interaction within their environment, children are active participants in 
the construction of their own knowledge. 

4. Jerome Bruner, like Piaget, was also interested in children’s stages of cognitive development but 
emphasized “Learning through Discovery.” Bruner felt that children were more likely to understand 
and remember concepts they discover in the course of their learning. 

5. Papert, an American student of Piaget has profoundly influenced the field of educational technology. 
He felt that students could advance in their intellectual abilities more quickly with the right kind of 
environment and assistance. Papert also believed that children should pl&y & big part in teaching 
themselves.” Papert was the developer of LOGO. (Roblyer & Edwards, 2000). 

Classroom Environment, Climate, and Management 

Classroom environment is a term that refers to a couple of very important dynamics within the 
educational context of a classroom: 1) the climate of the classroom, and 2) the management of the 
classroom. Classroom climate is a term that is used to refer to the teacher and classroom characteristics 
that promote student’s feelings of safety and security, together with a sense of success, challenge, and 
understanding. Classroom climate is important because it encourages both motivation and achievement. 
Students learn best in a general atmosphere that is safe and orderly and that promotes success on tasks of 
worth and substance (Eggen & Kauchak, 1997). In a healthy classroom climate, students are viewed and 
treated as competent people and they understand the requirements of learning tasks, perceive them as 
challenging, yet believe they will succeed if they make reasonable effort (Brophy & Rohrkemper, 1987). 
Classroom management refers to the combination of teacher strategies and classroom organizational 
factors that lead to a productive leaning environment. This includes established routines, classroom rules, 
teacher responses to student behavior, and instruction that promotes a climate conducive to student 
learning (Eggen & Kauchak, 1997). The characteristics and outcomes of effective classroom management 
are as follows: 
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❖ Increased achievement 

❖ Improved motivation 

❖ Preventive strategies for behavioral problems 

❖ Positive Physical environment 

❖ Establishment of procedures 

❖ Creating effective rules 

❖ A teacher that listens and cares 

❖ Communication with students 

Organization and Planning (Eggen & Kauchak, 1997). 



Pre- and Post-Instructor Activities 

To ensure that learners benefit the most from a virtual field trip, instructors would do well to familiarize 
themselves with the selected virtual field trip. This includes the subject/content area, but how the material 
is presented, learning activities, types and method of interaction, and what resources are provided. In 
addition, Cox-Peterson and Melber (2001) recommend orienting the students to the virtual field trip to 
“reduce the novelty effect (which) can hinder the learning process” (p. 1 8). Properly designed and 
implemented set induction is one method to accomplish this pre-instructor activity. Summaries and 
conclusions as well as “closing” activities address post instruction activities. Tie to the curriculum may be 
accomplished by means of print materials available from the virtual field trip content provider (where 
applicable). Teacher implemented age-appropriate closing activities can also be used to both integrate the 
virtual field trip into the curriculum as well as reinforce student learning. Cox-Peterson and Melber 
(2001) suggest the use of multimedia presentation, which both organizes and captures the essence of the 
virtual field trip. 



Lessons Learned 

Deploying virtual field trips in the K-8 learning environments requires a significant amount of preparation 
on the part of the instructor. Perhaps, one of the major concerns of educator is the quality and relevance of 
virtual field trips as this is solely under the control of the content provider(s). Therefore, an important 
initial step for educator is to thoroughly evaluate both the content and forms of interaction used by 
content providers. Relevance to lesson plans and intended learning objectives provide a sound basis for 
selecting a particular virtual field trip. Reactions to a particular virtual field trip from educators as well as 
students can be used to assess the overall impact. All things considered, the questions to ask oneself are 
similar to that of purchasing a textbook or attending a concert: “Would I pay to access this virtual field 
trip?” and “What measurable impact will this virtual field have on student learning?” 
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Mission Impossible: 

How to Design an Online Course Without a Team 

Penny Ralston-Berg 
Instructional Designer 
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Bethany Gordy 
InstructionalA^isual Designer 
University of Wisconsin Learning Innovations 

Although research shows that it takes a team of experts to efficiently develop a high quality online course, 
not all course designers have the opportunity to use a formal team approach. Due to sparse or scattered 
resources, budget limitations, time constraints, academic procedures and policies that may not fully 
accommodate distance learning, or other reasons outside their control, some online course designers find 
themselves tasked with developing an online course from start to finish with little or no help. While not 
the ideal situation, with planning and perseverance, a quality online course can be produced under these 
circumstances. 



Starting Points 

When beginning to develop online courses, course designers must honestly assess their own skills and 
make plans accordingly. In this model, it is assumed that the course designer is also the content expert. 
However, developing and facilitating an online course requires a variety of additional design, production, 
technology, and communication skills. 

Skill and experience related questions include: 

❖ Are you comfortable with technology in general? 

❖ Have you integrated Web technology into an existing face-to-face course? 

♦J* Have you created your own Web pages? 

❖ Do you have a basic understanding of plug-ins and how they are used with Web browsers? 

❖ Are you familiar with rules and protocol of online communication? 

❖ Have you facilitated threaded discussions? 

❖ Have you facilitated online chats? 

♦♦♦ Are you familiar with successful teaching and learning styles relative to Web-based instruction? 

❖ Are you familiar with your campus policies on issues related to online learning including Web 
accessibility, ADA compliance, and copyright guidelines? 

♦> Are you comfortable learning new things? 

*1* Do you know how to teach yourself new tasks in an efficient and effective way? 

❖ Can you set deadlines for yourself and complete tasks on time? 

*t* Are you familiar with library resources available to distance learning students? 

*** Can you easily view a piece of instruction from a student point of view? 

In addition to a self-assessment, online course designers must also consider how the course will be 
delivered to students. The delivery method may have a tremendous impact on the development and 
facilitation timeline. 
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Delivery related questions include; 

❖ What software or courseware is available? 

❖ Will you need to acquire new skills to use it? 

❖ Is there a course administrator? Or will I be required to enroll students? 

❖ How will students register for the course? 

❖ How will I get access to the online learning environment? 

❖ What are the technical requirements for developing course materials? 

❖ When will I have access to the learning environment? 

❖ Is technical help available for students? 

The answers to these questions and other skills and experiences will affect the time, budget, and in the 
end, the quality of the online course. If knowledge gaps are identified, consider potential sources of help 
and seek them out. Are there technology training, instructional design, or production resources available 
on campus? Are you really alone? 

It helps to make these contacts early in the process. Sharing your own goals, experiences, strengths and 
weaknesses with these contacts will also help make the most of available support and resources. If no help 
is available in certain areas, prepare and list of tasks or needed skills and build this need for acquisition of 
new knowledge into the timeline for the course. 

Planning and Timelines 

When planning, it is important to consider all the roles and tasks. Aside from content expert, other roles 
include; 



❖ Project manager 

❖ Interface designer 

❖ Instructional designer 

❖ Media developer 

❖ HTML author 

❖ Editor 

❖ Quality control analyst 

❖ Alpha-tester 

❖ Course administrator 

❖ Course facilitator 

❖ Technical support provider 

❖ Online library resources liaison 

❖ Self-motivator 

Planning is the phase when the realistic appraisal of skills and experience meets the timeline. A realistic 
timeline must be developed based on the self-assessments and information gathering mentioned above. 
There may be a need to delay the actual development of the course in favor of increasing skill sets and 
working out details of delivery. The plan may also include development tasks and professional 
development that occur simultaneously in order to meet the deadline for delivery. 

If development timelines are extremely short and help is very limited, it may also be necessary to drop 
extraneous media or programming pieces that are not essential to the instructional goals and value of the 
course. It is also important to add extra time for unexpected problems or steeper than expected learning 
curves. Media enhancements may be added to later versions of the course, allowing those course 
designers with little or no available help to build new skills over time and not to learn all possible skills in 
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one development cycle. This practice of gaining mastery of new skills over time will also add to the 
quality of ^e course. 



Course Design 

Course content can be presented in an online format through various types of media including text, audio, 
video, animation, presentation software, instructional graphics, etc. There are also a wide variety of 
communication technologies available including e-mail, online chat, threaded bulletin b(wds, etc. When 
selecting the media or tools to use within a course, it is important that selections match instructional goals 
and objectives, the technical capability of the target audience, and the resources available to the wurse 
designer. For those designers with little or no help, external, online sources of pre-produced media may 
be available. There are also low no-cost communication technologies available. 



Other design elements to consider include: 

♦J* Infuse your personality and teaching style into the course 
♦> Creating effective discussions by eliciting critical thmking 

❖ Break lengthy content pieces into smaller chunks 

♦J* Present complicated concepts or processes as informational graphics 

❖ Add FAQ (Frequently Asked Questions) pieces based on your experience with traditional 

classroom students 

♦> Remember that group work is still possible 

❖ View the course from a student’s point of view (or ask someone else to do so) 

❖ Remove as many questions or unlmowns as possible from learning activities 

❖ Be aware of and follow any copyright and fair use policies on your campus 

❖ Be aware of and follow any ADA or accessibility policies on your campus 



Again, extras add to development time and require advanced skill sets. Instructional goals and objectives 
within the course should be the first determining factor in the decision to develop additional media, 
graphics, interactive pieces, or communication-based activities for the course. A mixture of well- 
presented content enhanced with instructionally appropriate media and thought-provoking discussions can 
result in a satisfying and effective learning experience for online students. 

Facilitating Online Courses 

Just as in the traditional classroom, online courses require more than just a content expert. They require a 
motivational coach, discussion facilitator, mentor, and mediator. 

Other keys to facilitating online courses include: 

❖ Set communication ground rules early in the course 

Lead by example in online discussions; your participation encourages students to participate 

❖ Give feedback on a regular basis 

❖ Set clear expectations on when and how feedback will be given 

*t* Provide office hours for online students via bulletin board, e-mail, chat or phone 

❖ Provide the students with a master calendar or checklist of all learning activities and due dates 
♦J* Provide students with list of contacts by topic (content, tech support, registration, etc.) 

♦> Facilitate, mediate, mentor, and coach; be active and ever-present 

❖ Develop time-saving techniques for yourself 

❖ Try to anticipate technological glitches 

♦♦♦ Build redundant or alternative procedures into the course to accommodate these glitches 
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❖ Get preferred or most frequently used e-mail addresses from all students 

Evaluation in Online Courses 

Online course development is a continuous cycle of development and revision. To keep quality high, use 
evaluation results to drive revisions to the course. 

Ideas for evaluations include; 

❖ Treat the first run of the course as a pilot, gathering extensive data from participants 

❖ Online surveys before, during and after the course 

❖ Small surveys for individual activities within a course 

❖ Focus groups 

❖ In-person, e-mail, or phone interviews with former students 

Share Results and Experiences 

At the completion of the course development process, it is important to share results and experiences - 
both good and bad. Sharing helps online course designers to reflect and plan improvements for future 
courses. Inexperienced faculty can also benefit from lessons learned by others. 

Sharing can be accomplished locally through presentations, brown-bag informational sessions, or meeting 
with other online faculty throughout each school year to revisit lessons learned and share new 
experiences. As experience grows, some faculty may become mentors to others. Sharing can also bring 
visibility to the need for support of online course development and to the potential benefits of a team 
approach to online course development. 
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A Smile for SMIL: 

A Primer on Creating Cost-Effective SMIL Presentations 

Brandon C. Taylor, M.S. 
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DePaul University 

Multimedia materials for educational, informational and entertaining presentations can be created simply 
and economically with SMIL (Synchronized Multimedia Integration Language), a basic html-like mark- 
up language that can coordinate the timing and layout of streaming video, audio, images, animation, and 
text. One can create and access SMIL with ordinary hardware, a variety of free SMIL editing tools and 
modest dial-up network connections (e.g. 28.8 kbps) making it a low-cost, low bandwidth and effective 
streaming media option. 



Introduction to SMIL 

What Is SMIL 

SMIL (pronounced smile). Synchronized Multimedia Integration Language is a markup language similar 
to HTML (Hypertext Markup Language) that enables you to add tags to a text document so than you can 
easily combine and synchronize text, still images, animations, audio and video (Williamson, H., 2001). 
SMIL is a com. SMIL is standards based component of streaming media. SMIL is used to control various 
features of streaming media. 

History of SMIL 

The history of SMIL dates back to mid 1990’s. 

In October 1996, a small group representing the CD-ROM and Web multimedia communities 
gathered at a workshop hosted by the World Wide Web Consortium (W3C), a group of 
researchers and industry representatives that strives to advance the growth of the Web by 
developing standards. Their task was to explore the creation of a language that could be used to 
synchronize multimedia on the Web. Unlike the scripting languages in use for creating 
multimedia, this new language would be descriptive. It would not require a programmer to make 
Web multimedia (Kennedy, T. and Slowinski, M., 2002, p. 40). 

The W3C’s offered the first SMIL specification, SMIL 1.0 on 15 June 1998 (W3C, 2001, 
http://www.w3.org/TR/REC-smil/). Although, SMIL 1 .0 is relative simple to use in the creation of 
synchronize multimedia in web-based media, it is somewhat limited. Thus, the W3C offered the second 
and most current SMIL specification as of the press time of this paper, SMIL 2.0, on 7 August 2001 
(W3C, 2001 http://www.w3.org/TR/smil20/). Whereas, SMIL 1.0 is simple and easy to get into quickly, 
SMIL 2.0 is more complex - yet much more powerful (Kennedy, T. and Slowinski, M., 2002, p.44). 



Why SMIL 

SMIL based Web multimedia presentations, which can also be developed on portable media such as CDs, 
can be created with just a typical free text editor as well with the various free and for cost SMIL editors. 
Thus, SMIL is a low cost investment with a significant return on investment for Web-based, broadcast 
like multimedia presentations. Although, many may be more familiar with less utilitarian applications, 
however, SMIL has many utilitarian applications such as text captions for audio and video, multiple 
language presentations, etc. 
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The competitive market place will always offer a range of technologies to choose from. A 
solution that helps you meld the strengths of these technologies into a cohesive presentation is 
just that kind of Web multimedia we all need. SMIL is that solution. SMIL is “technology 
agnostic”, it allows Web multimedia technologies to collaborate with each other. For example, 
Macromedia Flash is a great vector-animating tool. It is acceptable for streaming audio but it is 
not so robust with streaming video. RealNetworks’ RealMedia is greate a streaming audio and 
video. Why not combine the strengths of each? SMIL allows that kind of collaboration (Kennedy 
& Slowinski, 2002, p.9). 

Thus, SMIL should not be seen as a competitor of other web-based technologies, but as a significant 
value added companion of other web-based technologies. 

How SIVHL Works 

SMIL basically works as a multimedia director, pulling together various multimedia components of 
various technologies in a television like presentation. SMIL orchestrates the timing, layout and 
hyperlinking of various multimedia in a given SMIL presentation. For example the following SMIL 2.0 
code simultaneously plays an instrumental version of a song with an acapella version of another song, 
thus creating a third song in a virtually invisible white background with modified volume levels: 

<!— This section is necessary to use the SMIL 2.0 features— > 

<smil xmlns = "http://www. w3. org/2001/SMIL20/Language " 
xmlns:m = "http://features. real, com/200 1/SMIL20/Extensions "> 

<!—This section, the header and layout section, describes the metadata and layout of the presentation— > 
<head> 

<meta name="Title" content="Cop the Math" /> 

<meta name=" Author" content="Brandon C. Taylor, MS." /> 

<meta name= "Description" content=" Educational, mathematical and lyrical remix of Youngin'" /> 
<meta name="Copyright" content="(c)2002" /> 

<layout> 

<root-layout width = "1 " height= "1 " background-color= "white " /> 

<region id= "vocals " soundLevel= "800% " left= "I " top="l " width="l " height^ "1 " z-index-^ "1 " 
background-color = "white "fit= "fill" /> 

<region id= "instrumental" soundLevel= "100% " left= "1 " top= "1 " width="l " height= "1 " z-index="l " 
background-color = "white "fit= "fill " /> 

</layout> 

</head> 

<!—The this section, the body section, presents the content — > 

<body> 

<par> 

<audio src="copthemath-a.rm" region= "vocals" abstract="vocals" /> 

<audio src="youngin-i.rm" region= "instrumental" abstract="trackl " /> 

</par> 

</body> 

</smil> 
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SMIL code resembles HTML code in many ways. Thus, just as a web browser is required to experience 
HTML web pages, SMIL players or certain browsers are required to experience SMIL presentations. 

Selected Free SMIL Players and Browsers 

Players and Browsers Overview 

Although, there are a number of SMIL players and browsers, I will focus on the free players and 
browsers. Today, the differences between a player and a browser are getting pretty fuzzy (Kennedy, T. 
and Slowinski, M., 2002, p. 10). A number of SMIL players and browsers also support experiencing other 
streaming media formats such as proprietary versions of SMIL code, audio/video files, etc. Some SMIL 
players do not support SMIL 2.0. 

Real Player (http://www.real.com/realone/index.html?lang=en&loc=us) 

SMIL players go, RealNetworks’ RealPlayer is one of the best-equipped web multimedia players around 
(Kennedy, T. and Slowinski, M., 2002, p. 10). RealPlayer has a version for Windows, Macintosh, Linux, 
and UNIX. RealPlayer . RealPlayer Basic is the free version of RealPlayer. SMIL support is found in 
RealPlayer G2 and RealPlayer G2 Basic and later. RealNetworks latest player as of the press time of this 
paper, RealOnePlayer supports SMIL 2.0. 

QuickTime(http://www.appIe.coni/quicktime/dowiiload/) 

Apple Computer’s QuickTime player versions 4. 1 and later support SMIL, as well as many other 
multimedia formats. QuickTime is available in Macintosh and Windows versions. 

Microsoft Internet Explorer 5.5 or later (http://www.microsoft.com/windows/ie/defoult.asp) 

Microsoft Internet Explorer 5.5, or later supports experiencing SMIL 2.0 presentations only in its 
Windows versions of Microsoft Internet Explorer (Kennedy, T. and Slowinski, M., 2002, p. 14) . 

Although, Microsoft has a streaming media product, Windows Media Player, its SMIL support is 
currently via its web browser as previously mentioned. 

SOJA (http://www.helio.org/products/smil/download/) 

SOJA is a Java based SMIL player developed by Helio, a small French non-profit organization. SOJA is 
best used to view SMIL presentations that do not contain proprietary versions of SMIL or other 
multimedia. 



Selected Free SMIL Editors 



Free SMIL Editors Overview 

Although, SMIL code can be create with just a text editor such as Microsoft Notepad, there are a number 
of free SMIL editor tools that can greatly facilitate creating SMIL. Various free SMIL editors have both 
advantages and disadvantages, thus using multiple free SMIL editors may facilitate one’s SMIL 
development. Also, some free SMIL editors create proprietary versions of SMIL presentations for a 
certain SMIL player. However, the ability to code SMIL without a SMIL editor is advantageous as the 
free SMIL editors are somewhat limited and thus may require manually coding some aspects of the SMIL 
code for various desired effects. Also, many of the free SMIL editors do not support SMIL 2.0. 
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Fluition (http://www.fluition.com/cgi-biii/dowiiload.cgi) 

Fluition, by Confluent Technologies, has a free demo version of its SMIL editor. Fluition is particularly 
inclined to creating SMIL presentations containing videos. Fluition is available in Windows and 
Macintosh versions. Fluition’s free version does not allow one to edit an existing SMIL presentation. 

RealSIideshow 

(http://foniis.real. CO m/mfoniis/products/tools/slideshowbasic/iiidex.htiiil?key=8J62Fl 024050138) 

RealNetworks’ RealSIideshow is a very intuitive tool and has a lower learning curve than most other 
SMIL editors. RealSIideshow Basic is the free version of RealSIideshow. RealSIideshow is only 
available for Windows and it is best suited for creating SMIL presentations of still images with transitions 
and audio tracks for each image as well as a background music/audio track. RealSIideshow also creates a 
Realnetworks proprietary version of SMIL. 

SMIL Composer (available on CD or via e-mail from an existing user) 

SMIL Composer is a full-featured freeware SMIL editor by Sausage Software. SMIL Composer seems to 
only be available on a companion CD that accompanies the SMIL for Dummies book written by Heather 
Williamson and published by IDG Books Worldwide, Inc. or via e-mail from someone that has the 
software file. SMIL Composer only has a Windows version. 

Presenter One (http://www.accordent.com/presenterone/) 

Presenter One, by Accordent, is a SMIL editor that facilitates creating streaming audio annotated 
PowerPoint presentations in RealNetworks Real Media format. Presenter One Standard is the free 
version and is limited such as it is only allows the creation of not more than fifteen minute presentations. 
Since RealNetworks and Accordent’s have a business relationship and that RealNetworks doesn’t have 
its RealPresenter software available on its web site, it seems that RealNetworks has discontinued 
RealPresenter, which had similar features as Presenter One. 

Magpie (http://ncam.wgbh.org/webaccess/magpie/mdex.html) 

Although, Magpie (Media Access Generator) is primarily a tool for creating text captions for digital video 
files, it can export the captioned files in to a RealNetworks proprietary version of SMIL. Magpie is 
produced by the National Center for Accessible Media (NCAM). 

Summary 

In summary, given the availability of various free SMIL editors, the fact that SMIL presentations can be 
created with just a text editor, the various resources for learning SMIL and the various features of SMIL, 
SMIL is a relatively low-risk, low investment with a potential high return on investment. 
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Selected SMIL Resources 



http://www.w3 .org/TR/smil20/ 

http://www.w3 . org/TR/REC-smil/ 

http://www.justsmil.com 

http://www.smilbook.com 

http://service.real.com/help/library/ 

http://ncam.wgbh.org/webaccess/magpie/index.html 

http://edtech3.cet.uiuc.edu/efantis/html/fsi2000/introsm/introsm.htm 
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Affordable desktop computer systems have made the delivery of asynchronous and synchronous distance 
teaching and learning utilizing powerful digital video (DV) within the reach of all educators. Educational 
research indicates that pictorial information is easier to remember than textual information (Anglin, 
Towers, & Levie, 1996; Braden, 1996) and so integrating video and audio technologies into distance 
learning environments can provide stimulating and powerful learning experiences that actively engage 
learners by exposing them to subject matter in different ways. However, the design, development, and 
implementation of digital video in today’s media-rich world poses many challenges and can be a 
confusing process because there are so many affordable technologies, options, mediums, and programs 
for producing, editing, and encoding video. In order to select appropriate technologies for digital video 
production, you need to be aware of the following basic information related to the three key stages 
involved in the creation of high-quality educational videos: Pre-production, Production, and Delivery. 

Pre-Production 

Pre-production planning will save time and help insure quality. There are several elements to consider 
before you begin filming or shooting video: Planning, budgeting and storyboarding. First, determine the 
software and hardware that you will need to produce the quality and quantity of video files that you 
desire. When you are thinking of what you want to accomplish with a video production some basic items 
to consider include: What is your target audience? What content is to be delivered? How will the final 
video be delivered to the learner? What instructional strategies will facilitate the learning to be 
accomplished? What locations will you need for filming? Next, consider what you need to collect and 
prepare prior to filming such as: actors, scripts, props, and costumes. Once you have determined the 
preceding requirements, it is time for you think about budgeting for filming. Based on the time and 
money that you have available, the items on your list may need to be modified. Prior to video filming, the 
content, audio, and video scripts must be prepared and storyboarded. Storyboarding is the process of 
creating comic book-like pictures of the various scenes you plan to film and displaying video shots in a 
visual time-line, which allows for modifications and changes before actual filming takes place. By 
storyboarding, you can get an idea of how the different film shots you plan will work together. 

Remember, the main purpose of a storyboard is to help you understand what shots you will need to film 
and what those shots will look like (Long, 2002). 

Production 

To film and produce digital movies, you will need to have equipment, ranging from a camcorder and 
computer, to lights, microphones, and cables. Even if you already have existing computer equipment, it is 
important to understand what you have, what you mi^t need, and how you will get all of the elements 
working together to edit and deliver digital video (Long, 2002). 
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Video Formats and Camcorders 



Two basic video formats are currently available: analog and digital. Telephones and broadcast television 
have conventionally used analog technology, which accomplishes electronic transmission by adding 
signals of varying frequency or amplitude to carrier waves of a given frequency of alternating 
electromagnetic current. Analog camcorders such as VHS, SVHS, and Hi-8 are widely available and less 
expensive than digital camcorders. However, digital signals are more efficient, well defined, and orderly 
than analog signals and are easier for electronic circuits to distinguish. In fact, a computer processor can 
analyze analog data only by converting it into a digital form by utilizing an electronic process to change 
the analog (continuously variable) signal into a digital (multi-level) signal. Digital camcorders, which 
utilize digital “MiniDV” tapes, encode video signals, “in digital bits for storage, transmission and 
display” (Agnew, 1999, p.2) and provide better image quality, power, and flexibility than do VHS, 
SVHS, and Hi-8 cameras. Michael Rubin explained what “high-quality video” means by utilizing a ten- 
point scale where 1 is low quality video and 10 is high quality video, “If the high-quality video that 
broadcasters use is a 10 and lousy Internet video is a 2, a typical VHS rental video is about a 3. S-VHS 
and Hi-8 offer great improvements over older consumer formats, and rate around a 5. MiniDV clocks in 
at a whopping 9.7. Mini-DV images are more than twice as good as anything consumers have had access 
to in the past” (2002, p. 3). 

There are many affordable digital camcorders available today. Fortunately, choosing one isn’t too hard 
after you determine the features you want and what you can afford. Your main concern when buying a 
camcorder should be image quality: Purchase the camera that will shoot the best image that you can 
afford. Digital camcorders utilize either 1-chip or 3-chip, charge-coupled devices (CCDs) to digitize 
video images. Three-chip cameras, which are used for broadcast television today, deliver better color and 
do a better job of reproducing fine edge detail than ICCD devices because a single chip camcorder uses a 
single imaging chip to grab a full-color image, while a 3CCD camcorder uses separate imaging chips for 
red, green, and blue information merging this more specific image information into a full-color file. When 
checking out a camera’s image quality, consider questions such as the following: How is the overall 
color? Do I need the camera to perform in low light? What quality of sound do I need? Most camcorders 
have built-in microphones which are suitable for some purposes, but for serious audio work you may 
want to attach an external microphone, if so, make sure that the camcorder has an external microphone 
jack and headphone jack for monitoring audio while recording (Long, 2002). There are many different 
types of microphones, but the most common are handheld microphones and clip-on lavaliere 
microphones. When recording individual voices, lavalieres are the best choice because they are small, 
unobtrusive, and deliver good quality while handheld microphones are best for recording dialog or group 
conversations. 

Desktop Digital Editing Systems 

You may wonder, which desktop computer configuration is best for digital video work? A desktop 
computer system must have enough processing power and storage space to digitize audio and video. 
Ideally, beginners should try to buy a computer that is already set up and pre-configured for video work. 
Macintosh based systems have an advantage over Windows based systems when it comes to digital video 
because Firewire video support is well integrated into all Apple computer systems, while Windows-based 
machines don’t usually have Firewire support built-in. Apple computers were the first to offer graphic 
software and photo manipulation; therefore, most photo-manipulation software was developed originally 
for this platform. Macintosh operating system works well with digital images and was the first consumer 
grade desktop computer to have high-speed graphic cards that offered millions of onscreen colors 
Macintosh computers have been widely used in much of the movie and TV industry, but Windows 
machines are gaining ground and there is now a very good selection of digital editing shareware and 
commercial software for both computer platforms (Jones, 1999). 
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When working with digital video more computer processing speed and storage space is better than less, so 
purchase as much processing power and hard drive space as you can afford. We recommend these 
minimum computer specifications for working with digital video: A Macintosh G3 or G4 Power PC or a 
Windows Pentium IV processor (minimum of 300MHz), 256 MB of physical RAM, a 60 GB hard drive, 
(any less than this and you are going to be frustrated) and a Firewire card. 

If you are shooting or capturing analog video you will need a converter to change analog signals into a 
digital format for editing. Analog video, which has been digitized, is not as high in quality as native 
digital video. Firewire, (IEEE 13 94) is a high-speed serial data connection or port, which is used by DV 
camcorders and desktop computers. National Television Standards Committee (NTSC) is the television 
broadcast video standard used primarily in North America and Japan, while the Phase Alternate Line 
(PAL) standard is used primarily in Europe, Asia, and southern Africa. Video files contain multiple 
streams, generally one video and one audio stream. DV captured by Firewire can be stored in two 
different formats. The whole DV stream, both audio and video, is stored as one unmodified large digital 
video stream in the first format. In the second format, the DV stream is split and stored as separate smaller 
video and audio data streams. MPEG (pronounced M-peg) stands for the Moving Picture Experts Group, 
which develops and maintains international digital video and audio encoding standards employing 
procedures for developing, adopting, and reviewing digital multimedia to insure that video files can be 
played by standards-compliant video software on different computer platforms. MPEG is also the 
nickname given to a family of international standards used for coding audio-visual information in a 
compressed digital format. 

Multimedia Architectures are plug-ins and system files that allow desktop computers to create, store, and 
play video files. Codecs is any technology for compressing and decompressing data. Codecs can be 
implemented in software, hardware, or a combination of both. Algorithms handle the compressing and 
decompressing of digital video for the different architectures. Different architecture have special codecs 
built-in, such as QuickTime, Windows Media, and RealMedia and some codecs are common to different 
architectures. Common video codecs include: Cinepak, desired for playing small clips on computers 
with single-speed CD-ROM drives; Motion- JPEG, used mainly for editing and video storage rather than 
playback; MPEG-1, developed for VMS quality on CD ROM; MPEG-2, a standard developed for digital 
TV, DVD-ROM and HDTV; MPEG-4 developed for the delivery of Web video; and Sorenson Video 
developed for high quality CD-ROM and Web applications. 

Most of today’s camcorders capture video at 29.97 frames per second. This odd number has to do with 
technological changes that were made when color television was invented. Video originally vvas filmed at 
thirty frames per second for black and white television. Each frame of video is numbered using a 
continuous track time code during filming. If you pause the camcorder and start shooting again, the 
camera picks up the time code where it left off. However, if you stop the camcorder between shots, breaks 
in the time code can occur, which makes later digital film editing more complicated. When you purchase 
new video film cassettes it is a good idea to “stripe” a continuous time code on the tape. This can be 
accomplished by recording the tape from beginning to end with the lens cover in place. 

Filming is the key to successful movie-making process. Basic guidelines to ensure usable and reusable 
video footage include: Compose your shots carefully, use a tripod to help you to steady your shots, and be 
careful not to overuse zoom controls which can cause problems with focus. While you are shooting 
video, you may encounter problems with low battery power. We suggest that you use the camcorders 
AC adapter during filming whenever possible. When you must use battery power you can extend the 
battery life by not using the LCD screen, not searching around on the tapes and changing tapes only when 
necessary, not rewinding and reviewing your footage, turning the camera off when you are not filming, 
and keeping the batteries warm if you are filming in extremely cold conditions. 
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Post Production 



After filming you will need to use special software and hardware to edit, produce, and deliver digital 
movies. Digital editing software allows the user to transfer video clips to the computer, edit these clips, 
and then output them back to videotape, CD, DVD, and for the Web. Apple’s iMovie, which currently 
ships with all new Apple computer systems, is a fully-fimctional, simple to learn and user-friendly digital 
editing software package, which can also be purchased separately online. There are more powerful and 
expensive professional level digital editing programs for personal computers such as Adobe Premiere, 
which is available for both Windows and Macintosh systems. A disadvantage of these powerful 
professional programs, in addition to being costly, is the fact that they take more time to learn and are 
more complicated to use. Long says that more advanced professional digital editing packages do offer 
editing tools that go far beyond the simple tools provided by simpler programs such as iMovie and 
suggests that if you have a choice, start out with iMovie, and work your way up to more serious 
programs, when and if you find that you want or need more powerful features (Long, 2002). 

Uncompressed video files are huge and need to be compressed (made smaller) before being delivered to 
your target audience. Another way to provide smaller files is if you have ten minutes of video that you 
want to put on the web for low-end machines, you can chop it up into smaller chunks to make it faster and 
more efficient for your audience to download the files. Another strategy for Web video involves setting 
the data rate of digital movies slightly lower than the throughput of the user's connection if you want them 
to be able to watch your movies in real time. For a 28.8 modem, that means a data rate somewhere around 
two KBps, for ISDN around five KBps, and for T1 lines from five to 40 KBps. To deliver video at low 
data rates compromises are often made. The image size must be small, the frame rate low, and the sound 
compressed, which may make the image quality less than optimal. Nonetheless, there is still interesting 
video and sound, which can be delivered using the Web (Lynch & Holton, 1997) when the digital video is 
properly prepared. It is critical to assess the intended audience, their network access, computer processing 
power and memory configuration, and planning for delivery accordingly. Once a digital movie is edited, 
the final step involves selecting the delivery medium that you will use to supply the video to the target 
audience such as videotape, CD, DVD, or Web streaming, and finally, delivering the video. 

Note: The Digital Video Producers, a group of video professionals at the University of Texas, Austin, 
headed by Diane Gierisch, Information Technology Services, is developing guidelines for digital video 
production and a complementary “Digital Video Guidelines” Web site. The first author of this paper may 
be contacted for additional information. 
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